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EXECUTIVE  SUMMARY 


The  objective  of  this  paper  is  to  examine  the  utility  of  embedded  training  (ET)  as  a 
contribution  to  the  Quadrennial  Defense  Review.  The  following  five  points  summarize  the 
findings: 

1.  The  purpose  of  ET  is  to  provide  effective  training  to  operational  units  and 
individuals  using  operational  equipment  ET  requires  the  ability  to  provide 
relevant  threats  and  scenarios,  assessment  of  performance,  feedback  for 
lessons  learned,  and  collection  and  management  of  data  on  unit  training.  The 
“embedded”  aspect  concerns  the  incorporation  of  a  training  capability  in  the 
operational  equipment.  The  choice  of  whether  to  embed  training  must  be 
evaluated  on  a  case-by-case  basis,  depending  on  the  feasibility  and  costs  of 
incorporating  training  in  the  operational  equipment. 

2.  The  main  advantage  of  ET  is  the  ability  to  train  with  the  same  operational 
equipment  that  the  unit  takes  to  war.  Training  can  take  place  at  home  station  or 
while  deployed.  This  type  of  training  has  high  fidelity,  is  concurrent  with  the 
latest  upgrades  to  the  equipment,  can  be  linked  to  large-scale  exercises  as  the 
capability  of  the  Defense  Simulation  Internet  (DSI)  grows,  and  permits  mission 
rehearsal  and  assessment  of  force  readiness.  Properly  analyzed  and 
configured,  ET  systems  are  intended  to  enhance  the  use  and  effectiveness  of 
the  operational  hardware. 

3.  The  disadvantage  of  ET  is  that  it  may  reduce  the  capability  of  operational 
equipment  by  adding  weight,  reducing  space,  and  increasing  wear  and 
maintenance.  Furthermore,  when  ET  functions  are  being  used,  the  capabilities 
of  the  operational  equipment  may  become  compromised  or  unavailable. 
However,  many  of  the  reported  disadvantages  are  often  the  result  of  inadequate 
planning  and  implementation  and  are  not  the  necessary  result  of  an  effort  to 
embed  training. 

4.  According  to  user  reports  identified  in  our  review  of  the  literature,  ET  is 
effective  and  generally  acceptable.  Nevertheless,  litde  valid  and  reliable 
empirical  data  exist  on  ET’s  cost  and  effectiveness.  An  effort  to  collect  this 
information  is  needed. 

5.  Army  and  Navy  Service  guidance  has  promoted  ET  as  the  preferred  training 
alternative  for  developing  systems.  Defense  policy  should  continue  to  favor 
ET  as  a  requirement  in  new  weapon  systems  until  it  is  ruled  out  in  any 
particular  case  by  careful  review  of  its  probable  effectiveness  and  cost.  Such 
consideration  from  the  start  of  new  programs  offers  the  best  possibility  of 
training  that  will  optimize  the  performance  and  effectiveness  of  new  weapons. 
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THE  UTILITY  OF  EMBEDDED  TRAINING 


A .  OBJECTIVE  AND  ORGANIZATION 

This  document  represents  a  contribution  to  the  Quadrennial  Defense  Review  by 
examining  the  utility  of  embedded  training  (ET)  for  military  training.  We  begin  by  defining 
ET  and  identifying  different  types  of  ET  that  have  been  implemented.  Next,  we  examine 
past  and  present  interest  in  the  subject  and  identify  advantages  and  disadvantages  of  this 
training  method.  We  then  assess  new  technologies  that  relate  to  ET,  examine  the  research 
on  training  effectiveness  and  costs,  and  identify  related  areas  of  research.  Finally,  we  pre¬ 
sent  our  conclusions  about  this  approach  to  training. 

The  appendixes  contain  additional  information  on  this  topic. 

B .  DEFINITION  OF  EMBEDDED  TRAINING 

ET  refers  to  the  use  of  operational  equipment  to  train  personnel  assigned  to  operate 
and  maintain  that  equipment  ET  relies  on  built-in  or  easily  added-on  components,  such  as 
computer-based  tutorials,  simulation  scenarios,  and  software,  to  sequence  material  appro¬ 
priately  and  to  score  performance  for  feedback  and  lessons  learned.  This  definition  is 
similar  to  definitions  used  by  the  Department  of  Defense  (DoD)  and  the  military  Services. 

Although  the  definition  of  ET  is  simple  and  straightforward,  it  lacks  precision 
because  several  distinctly  different  training  devices  and  technologies  can  be  classified  as  ET 
systems.  To  clarify  the  definition  somewhat,  training  researchers  and  developers  have 
identified  homogeneous  subcategories  or  “types”  of  ET  systems. 

The  following  types  of  training  technologies  are  commonly  considered  as  examples 
of  ET.  This  scheme  is  not  intended  to  provide  a  definitive  classification  scheme;  rather,  it 
is  meant  to  illustrate  the  range  and  variety  of  training  technologies  that  are  considered  to  be 
ET  systems.  Appendix  A  presents  more  detailed  descriptions  of  each  type,  along  with 
some  example  systems  and  unique  cost-effectiveness  considerations. 

•  Fully  embedded  systems.  The  prototype  ET  system,  called  “fully 
embedded,”  has  all  its  training  functions  built  into  the  operational  weapons 
system. 
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•  Strap'On  (or  appended)  systems.  Some  ET  systems  use  hardware  that  is 
added  to  or  connected  by  cables  (umbilical  connections)  to  the  operational 
equipment.  This  hardware  can  be  planned  components  of  the  proposed  opera¬ 
tional  equipment  or  retrofitted  to  existing  equipment. 

•  Practice  devices.  Some  ET  systems  provide  practice  to  the  exclusion  of 
other  training  functions.  These  systems  are  usually  supplemented  with  instruc¬ 
tors  or  other  automated  systems  to  compensate  for  their  limited  capabilities. 

•  Embedded  computer-based  tutorials.  These  systems  are  embedded  into 
general-purpose  computers  for  imparting  critical  knowledge  (as  opposed  to 
skills).  Such  tutorials  are  usually  embedded  in  operational  equipment  that  use 
general-purpose  computers  to  support  complex  decision-making. 

•  Embedded  job  or  performance  aids.  These  aids  differ  from  other  ET 
systems  because  their  primary  function  is  to  support  performance  on  the  job — 
not  to  train  in  preparation  for  job  performance.  Nevertheless,  embedded  job 
aids  and  other  ET  systems  share  a  common  goal  (performance  improvement), 
are  developed  from  human  front-end  analyses,  and  use  similar  technology  for 
development  (Zachary  and  Hicinbotham,  1994).  Examples  include  embedded 
aids  for  maintenance  tasks,  embedded  helps,  and  embedded  decision  aids  for 
complex  tactical  tasks. 

•  Certain  stand-alone  systems.  Stand-alone  systems  are  usually  presented 
as  alternatives  to  ET.  However,  certain  instances  of  these  systems  are  related  to 
the  concept  of  ET  and  are  even  regarded  as  legitimate  instances  of  ET.  This 
category  includes  stand-alone  systems  that 

-  are  installed  on  ships,^ 

-  can  be  linked  to  embedded  training  systems  by  Distributed  Interactive 
Simulation  (DIS)  technology,  and 

-  are  demonstrations  of  technologies  that  eventually  may  be  embedded  in 
operational  systems. 

Strasel,  Dyer,  Roth,  Alderman,  and  Finley  (1988)  stated  that  ET  systems — despite 
their  physical  differences — ^are  similar  because  they  share  certain  common  training  features 
or  functions.  Elaborating  on  their  definitions,  we  propose  that  the  following  training  func¬ 
tions  are  (or  should  be)  common  to  all  ET  systems: 


1  Stand-alone  training  systems  on  ships  are  sometimes  referred  to  as  “organic”  training  systems  (Stratton 
et  al.,  1996). 
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•  presenting  appropriate  threat  scenarios  or  other  relevant  information  that  could 
occur  before  or  during  combat, 

•  assessing  performance, 

•  providing  feedback,  and 

•  collecting  and  managing  training  records. 

Any  evaluation  of  the  effectiveness  of  ET  technology  must  consider  the  extent  to 
which  it  addresses  these  essential  training  functions.  These  training  functions  also  help 
clarify  the  definition  of  ET.  Some  writers  have  considered  training  rounds  and  training 
missiles  to  be  examples  of  ET  since  they  aid  training  in  the  actual  equipment.  However, 
such  training  aids  do  not  address  these  basic  functions  and  are,  therefore,  not  legitimate 
examples  of  ET. 

For  the  basic  training  functions,  ET  systems  do  not  differ  from  their  stand-alone 
counterparts — that  is,  training  aids,  devices,  simulations,  and  simulators  (TADSS).  The 
unique  advantage  of  ET  is  that  the  training  subsystems  are  continued  in  the  operational 
equipment,  which  is  issued  to  all  units  and  is  the  material  that  these  units  take  to  war. 
Moreover,  ET  provides  the  capability  to  use  this  equipment  in  its  most  relevant  way  at 
home  station  or  while  deployed  and  avoids  the  artificiality  of  going  to  school  or  training  on 
representative — ^but  not  real — equipment 

C.  MILITARY  INTEREST  IN  EMBEDDED  TRAINING 

Because  of  these  unique  advantages,  the  military  has  had  a  continuing  interest  in 
ET.  As  described  below,  this  interest  has  developed  over  several  years. 

1 .  Early  Embedded  Training  Systems 

ET  is  not  a  new  concept  One  of  the  largest  and  most  successful  examples  of  ET 
was  the  Semi-Automatic  Ground  Environment  (SAGE)  system  that  provided  training  for 
the  Air  Force’s  Air  Defense  Direction  Centers  (ADDCs),  starting  in  the  late  1950s.  Located 
at  12  sites  across  the  continental  United  States  and  Canada,  the  ADDC’s  mission  was  to 
detect  enemy  nuclear  bombers  and  to  direct  aircraft  and  missiles  to  intercept  them. 
Embedded  in  the  SAGE  system  was  the  capability  to  train  operators  by  providing  synthetic 
enemy  targets  to  the  radar  screens.  This  work  started  as  a  series  of  experiments  in  team 
training  at  RAND  (1952-1954)  and  led  to  the  establishment  of  the  System  Development 
Corporation.  This  company  developed  the  training  program  and  supplied  the  “raids”  used 
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for  nationwide  on-the-job  training  at  the  actual  direction  centers  and  combat  centers  that 
were  netted  to  each  other  (Parsons,  1972).  SAGE  developers  did  not  refer  to  their  system 
as  “embedded  training,”  because  the  term  had  not  yet  come  into  use. 

Another  example  that  dates  from  the  early  1960s  is  the  Training  Alarm  Controller 
(TAC)  that  was  used  to  train  fire  control  teams  of  Polaris  submarines.  TAC  was  a  fully 
embedded  system  that  emulated  failures  in  the  operational  Fire  Control  System  (FCS) — 
failures  that  could  occur  during  the  countdown  before  firing  a  real  missile.  The  TAC  acti¬ 
vated  the  actual  relays  in  the  FCS  to  provide  appropriate  inputs  to  the  fire  control  team 
(Annett,  1990).  This  ET  system  provides  an  example  of  “stimulation”  (versus 
“simulation”),  where  a  signal  is  injected  at  or  near  sensor  elements  so  that  the  entire  opera¬ 
tional  system  is  used  as  it  would  be  used  to  process  a  real  signal  (Hoskin,  Jorgensen, 
Manglass,  and  Reynolds,  1989). 

2.  Recent  Interest  in  Embedded  Training 

Since  the  1950s,  numerous  weapon  systems  have  incorporated  ET  technology. 
Hundreds  of  current  systems — many  not  specifically  identified  with  ET — ^have  some  ET 
capability.  In  the  last  10  years  in  particular,  the  interest  in  ET  has  increased  because  of 
improved  technical  feasibility,  a  constrained  military  budget,  and  an  interest  in  reducing  the 
costs  associated  with  training.  Training  costs  for  simulators,  schools,  travel,  and  operating 
tempo  are  increasingly  visible  items  in  the  budget  and,  consequently,  have  become  targets 
of  opportunity.  Training  decision-makers  increasingly  regard  ET  as  a  way  of  reducing 
costs  by  incorporating  a  training  capability  in  the  equipment  issued  to  all  combat  and  sup¬ 
port  units.  In  this  context,  the  Quadrennial  Defense  Review  offers  a  timely  opportunity  to 
review  the  utility  of  ET  technology. 

In  addition  to  these  general  economic  interests  in  ET,  four  specific  developments 
have  increased  the  relevance  of  ET  as  a  method  for  delivering  training. 

a.  Computer  and  Simulation  Technology 

Simulation  and  computer  hardware  technologies  have  improved  tremendously  in  the 
recent  past  (less  than  10  years).  Improvements  in  simulation  make  ET  increasingly  invisi¬ 
ble  to  the  user  and  maintainer,  and  DIS  standards  make  it  possible  to  connect  ET  systems 
with  other  training  devices  within  and  among  units.  Micro-miniaturization  of  electronic 
components  has  mitigated  the  oft-cited  problem  that  ET  adds  too  much  weight  or  takes  up 
too  much  space  in  the  operational  equipment  Software  improvements  have  also  enabled 
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the  development  of  training  functions  that  do  not  impair  or  interfere  with  operational  func¬ 
tions  of  the  weapon  system.  Not  incidentally,  improvements  in  materials  development  and 
manufacturing  have  resulted  in  significant  cost  reductions  for  high-technology  components. 
Thus,  many  ET  systems  significantly  more  capable  and  less  expensive  than  comparable 
systems  designed  only  5  to  10  years  ago. 

b.  Instructional  Methods 

Parallel  improvements  have  been  realized  in  instructional  methods.  For  instance, 
researchers  developed  detailed  guidance  on  the  incorporation  and  design  of  ET  systems 
(e.g.,  Finley,  Alderman,  Peckham,  and  Strasel,  1988;  Witmer  and  Knerr,  1991).  Innova¬ 
tions  in  instructional  procedures,  such  as  intelligent  tutoring  systems,  have  been  embedded 
in  operational  systems  (e.g.,  Williams  and  Reynolds,  1990;  Gluckman  and  Willis,  1994; 
P.J.  Moskal,  P.D.  Moskal,  Carolan,  and  Chatham,  1994).  In  addition,  several  commercial 
authoring  systems  that  potentially  decrease  the  time  and  costs  related  to  the  development  of 
embedded  tutoring  systems  and  decision  aids  have  become  available. 

c.  Training  Design  and  Development 

Training  designers  and  developers  have  become  increasingly  motivated  to  endorse 
ET  systems  as  a  preferred  method  of  training.  There  are  at  least  two  reasons  for  this  trend: 

1.  The  adoption,  or  even  simply  the  consideration,  of  ET  ensures  that  training  is 
incorporated  early  in  the  system  development  process.  Early  consideration  of 
training  is  a  major  principle  for  those  who  ascribe  to  a  systems  approach  to 
training  development. 

2.  ET  is  more  likely  to  stay  current  with  system  upgrades  than  traditional  training 
methods,  which  are  separate  from  the  upgrading  process. 

d.  Military  Strategy 

DoD’s  Force  Projection  Strategy  demands  that  military  units  be  able  to  deploy 
rapidly  with  minimal  preparation.  With  the  adoption  of  ET,  units  deploy  their  training 
capability  along  with  their  equipment  This  allows  units  to  train  in  theater  and  even  on  their 
way  to  the  theater.  During  Operation  Desert  Storm,  Marine  Corps  units  were  able  to  train 
with  the  Marine  Corps  Full-Crew  Interactive  Simulation  Trainer  (MCFIST) — an  appended 
ET  system  for  Ml  tanks — as  they  traveled  by  ship  to  their  deployment  site.  Although  this 
example  may  not  be  representative,  it  points  to  the  fact  that  ET  systems  are  consistent  with 
the  Force  Projection  Strategy  by  providing  training  to  deploying  and  deployed  units. 
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3 .  Military  Guidance 

Current  interest  in  ET  has  culminated  in  explicit  guidance  by  the  U.S.  military  on 
how  and  when  ET  technology  should  be  used.  Appendix  B  contains  an  annotated  listing  of 
ET-related  policy  statements  presented  by  the  DoD  and  the  military  Services. 

The  Army  and  Navy  Service  policy  has  favored  ET  as  the  preferred  training  tech¬ 
nology.  For  example,  the  Navy’s  Tactical  Training  Manual  for  the  Atlantic  and  Pacific 
fleets  (CINCLANTFLT/CINCPACFLT,  1996)  states  that  “. . .  wherever  possible,  conduct 
training  on-board  ship  with  organic  training  devices  and  installed  equipment  under  supervi¬ 
sion  of  shipboard  training  teams”  (Chapter  5).  The  Army  policy  is  even  more  clearly  in 
favor  of  ET.  In  1987,  General  Maxwell  Thurman  (then  Vice  Chief  of  Staff  of  the  Army) 
and  James  Ambrose  (then  Under  Secretary  of  the  Army)  issued  a  memorandum  estab¬ 
lishing  ET  as  the  preferred  training  technology  for  developing  weapon  systems.  This  con¬ 
cept  has  been  sustained  in  succeeding  Army  documents  and  guidance: 

•  Army  Regulation  350-38,  Training  Device  Policies  and  Management,  dated 
15  October  1993 

•  Training  and  Doctrine  Command  (TRADOC),  Embedded  Training  Action  Plan, 
dated  11  October  1994,  and  a  later  version,  3  June  1996,  including  various 
messages  related  to  TRADOC  Policy  (30  March  1995) 

•  Force  XXI,  Warfighter  XXI,  dated  9  June  1995 

•  WarNet  XXI,  The  WarNet  Action  Plan,  dated  13  August  1996. 

From  1988  to  1991,  the  Army  Research  Institute  (ARI)  issued  a  10-volume  guide 
for  implementing  ET  in  developing  weapons  system  (Finley,  Alderman,  Peckham,  and 
Strasel,  1988,  and  others).  Witmer  and  Knerr  (1991)  developed  abbreviated  guidance 
aimed  specifically  at  the  training  developer;  a  second  edition  was  issued  in  July  1996.  The 
other  Services  also  developed  similar  guidance  during  this  period; 

•  The  Navy  published  Lessons  Learned  From  Currently  Fielded  Navy 
Embedded  Training  Systems  (Hoskin  et  al.,  1989) 

•  The  Air  Force  published  Research  and  Development  Strategies  for  Embedded 
Training  (Walch,  Yee,  and  Burright,  1991) 

•  Warm,  Roth,  Sullivan,  and  Bogner  (1988)  conducted  a  tri-Service  review  of 

ET:  Tri-service  Review  of  Existing  System  Embedded  Training  (ET) 

Components. 

Clearly,  there  is  no  lack  of  literature  on  the  need  for  and  guidance  on  how  to  develop  ET. 
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D.  ADVANTAGES  AND  DISADVANTAGES 

Implicit  in  the  concept  of  ET  is  the  assumption  that  ET  systems  offer  some  distinct 
advantages  over  other  methods  of  training.  Thus,  we  identify  some  of  ET’s  advantages 
and  disadvantages  that  have  been  cited  in  the  literature.  Some  of  the  advantages  and  disad¬ 
vantages  are  unique  to  ET,  whereas  others  are  shared  with  other  training  approaches. 
Also,  we  present  the  disadvantages  not  so  much  to  argue  against  the  use  of  ET  but,  rather, 
to  suggest  limiting  conditions  for  the  implementation  of  ET. 

Table  1  presents  the  advantages  and  disadvantages  of  ET  that  relate  primarily  to 
training  effectiveness.  Some  of  the  disadvantages  could  be  interpreted  as  mirror  images  of 
advantages.  For  instance,  the  high-fidelity  of  using  actual  equipment  is  tempered  by  the 
operational  constraints  of  this  equipment.  We  indicate  such  complementary  relationships 
by  lining  up  advantages  and  disadvantages  where  appropriate. 

Table  2  presents  the  advantages  and  disadvantages  of  ET  that  relate  to  cost  issues. 
The  cost  issues  provide  fewer  complementary  relationships  than  the  effectiveness  issues. 

E.  ASSESSMENT  AND  DISCUSSION 

We  assessed  the  utility  of  ET  technology  based  on  materials  and  information 
retrieved  from  the  Defense  Technical  Information  Center  (DTIC)  and  the  World  Wide  Web. 
From  these  sources,  we  obtained  and  reviewed  59  technical  reports  and  related  documents, 
which  are  listed  in  the  bibliography  in  Appendix  C.  We  also  consulted  colleagues  in  the 
military  Services  to  identify  omissions  and  inaccuracies. 

We  identified  56  different  ET  systems  used  by  the  three  military  Services  and  joint 
Services  and  briefly  summarized  each  in  Appendix  D.  This  list  includes  significant  histori¬ 
cal  systems  that  are  no  longer  used  and  proposed  systems  that  have  yet  to  be  implemented. 
The  current  status  of  many  systems  could  not  be  determined,  so  the  number  of  ET  systems 
currently  in  use  is  not  known.  More  importantly.  Appendix  D  does  not  contain  information 
on  the  numbers  and  types  of  DoD  Service  members  trained  to  determine  the  total  training 
requirement  being  met  by  ET  systems.  On  the  other  hand,  the  appendix  provides  a  rela¬ 
tively  large  and  representative  sample  of  military  ET  systems  from  which  we  can  derive 
some  generalizations  about  the  state  of  ET  technology. 

Each  entry  in  Appendix  D  includes,  at  a  minimmn,  a  brief  functional  description  of 
the  ET  system  and  the  identification  of  the  operational  equipment  on  which  the  ET  system 
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Table  1.  Advantages  and  Disadvantages  of  ET 
Related  to  Training  Effectiveness 


Advantages 

Disadvantages 

Provides  high-fidelity  training  environrnent  (i.e., 
operational  equipment)  that  promotes  transfer  of 
training 

Fidelity  limited  by  operational  constraints  to  certain 
signals  and  responses. 

Promotes  increased  realism  in  home  station 
training 

Realism  limited  by  ability  to  mimic  actual  inputs  at 
home  station. 

Provides  complex  environment  appropriate  for  sus¬ 
tainment  or  continuation  training 

Complex  environment  may  not  be  appropriate  for 
initial  training. 

Training  system  is  deployed  with  warfighters 

Training  system  is  not  available  if  personnel  and 
equipment  are  deployed  separately. 

Training  system  is  available  or  accessible  to  war¬ 
fighter  at  home  station. 

Accessibility  to  ET  system  could  be  limited  for 
security  consideration,  maintenance  consider¬ 
ations,  interference  with  operational  capabilities,  or 
excessive  set-upAear-down  time. 

Promotes  consideration  of  training  issues  during 
design  of  operational  system 

Full  advantage  may  not  accrue  to  strap-on  ET  sys¬ 
tems  that  are  added  after  initial  design. 

Ensures  that  training  stays  current  with  equipment 
changes 

Advantage  may  not  accrue  to  strap-on  ET 
systems. 

Provides  job  aiding  and  decision  support  as  well  as 
training  functions,  e.g.,  coaching  during  mainte¬ 
nance  troubleshooting,  and  rehearsing  and  war¬ 
gaming  courses  of  action  in  command,  control, 
communications,  and  Intelligence  {C3I)  systems 

May  degrade  performance  if  job/decision  support 
aids  become  unavailable  (i.e.,  a  performance  aid 
becomes  a  performance  “crutch”). 

Catastrophic  consequences  for  Incorrect 
responses  are  minimized  if  operational  and  training 
modes  are  reliably  differentiated 

Potential  catastrophic  consequences  exist  for 
incorrect  responses  if  operational  and  training 
modes  are  not  reliably  differentiated. 

Provides  systematic  variety  of  scenarios  related  to 
unit's  mission 

Effectiveness  limited  if  capability  has  not  been 
provided. 

Provides  performance  assessment  and  feedback 

Effectiveness  limited  if  capability  has  not  been 
provided. 

Permits  rehearsal  of  assigned  missions 

Effectiveness  limited  if  capability  has  not  been 
provided. 

Permits  assessment  of  unit  readiness 

Effectiveness  limited  if  capability  has  not  been 
provided. 

Can  provide  training  at  all  levels,  from  crew  to  unit 
to  joint 

Effectiveness  limited  if  capability  has  not  been 
provided. 

Provides  component  media  for  integrating  live, 
virtual,  and  constructive  simulation  from  unit  to 
world-wide  levels 

Effectiveness  limited  if  capability  has  not  been 
provided. 
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Table  2.  Advantages  and  Disadvantages  of  ET  Related  to  Cost 


Advantages 

Disadvantages 

Reduces  or  eliminates  costs  relative  to  using  a 
simulated  training  facility 

Increases  system  development  costs 

increases  utility  of  operational  equipment  by  ena¬ 
bling  its  use  during  idle  periods 

Increases  system  development  and  maintenance 
costs 

Incurs  little  or  no  additional  costs  for  computer- 
based  equipment 

Increases  system  development  costs 

Reduces  requirement  for  stand-alone  simulators 

Increases  system  development  costs 

Total  costs  may  be  reduced  by  the  positive  impact 
of  the  early  incorporation  of  ET  on  equipment 
design  and  interface  issues 

Increases  system  development  costs 

Reduces  costs  related  to  operating  tempo  (e.g., 
flying,  driving,  steaming,  shooting) 

May  increase  wear  and  tear  on  operational 
equipment 

Can  have  positive  effect  on  system  reliability, 
availability,  and  maintenance  (RAM)  character¬ 
istics  by  improving  maintenance  testing  and 
training 

Can  have  negative  effect  on  system  RAM  charac¬ 
teristics  by  increasing  system  usage  and 
complexity 

Decreases  personnel  required  for  sustainment 
training 

Increases  personnel  required  to  maintain  ET 
systems 

Life-cycle  costs  of  training  and  training  equipment 
visible  in  system  concept  and  design 

Funds  allocated  to  training  subsystem  are  vulner¬ 
able  if  cost  overruns  or  spending  cuts  occur  during 
system  development 

Increases  costs  for  ^'ruggedizing”  ET  system 

Adds  weight  and/or  take  up  space  in  operational 
equipment 

May  require  additional  or  auxiliary  equipment  (e.g., 
generators) 

Cost  analysis  of  ET  must  include  life-cycle  equip¬ 
ment  costs  that  are  not  directly  related  to  training. 

Costly  to  modify  training  courseware  that  is  tied  to 
tactical  software 

is  implemented  (the  “parent”  system).  In  most  cases,  the  ET  system  has  a  name  separate 
from  its  parent  system,  but,  in  some  cases,  the  ET  system  is  not  distinguished  from  the 
system  in  which  it  is  embedded.  As  described  below,  some  ET  systems  had  additional 
information  concerning  user  assessments  of  the  system  and  positive  features  of  the  system. 

In  addition  to  identifying  the  details  of  specific  ET  systems,  our  review  revealed 
some  generalities  about  ET  methods.  These  generalities  address  issues  related  to  technol¬ 
ogy,  effectiveness,  and  costs. 
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1 .  Technological  Opportunities 

ET  is  not  a  unitary  technology  but  rather  a  diverse  collection  of  technologies  that 
address  a  common  purpose:  training  on  operational  equipment.  Emerging  technologies,  in 
particular,  have  provided  opportunities  for  new  concepts  in  ET.  Examples  include: 

•  The  On-Board  Electronic  Warfare  Simulator  (OBEWS).  The 
OBEWS  uses  an  electrically  erasable,  programmable  read  only  memory 
(EEPROM)  module  to  generate  in-flight  threat  scenarios  and  to  record  per¬ 
formance  for  debriefing  on  the  ground. 

•  The  World-Wide  Military  Command  and  Control  System 

(WWMCCS).  The  WWMCCS  is  implemented  on  general-purpose  com¬ 
puters.  These  computers  provide  an  inexpensive  platform  for  implementing 
embedded  tutorials. 

•  The  AEGIS  Combat  Training  System  (ACTS).  ACTS  incorporates 
local  and  wide  area  networking  of  systems  to  allow  homogeneous  teams  and 
disparate  units  to  practice  in  realistic  combat  exercises. 

•  The  Organic  Combat  System  Training  Technology  (OCSTT). 
OCSTT  incorporates  DIS  technology,  which  allow  this  ET  system  to  interact 
with  other  live,  virtual,  and  constructive  simulations. 

2 .  Effectiveness 

The  effectiveness  of  ET  is  properly  defined  as  the  extent  to  which  an  ET  system 
results  in  or  contributes  to  improved  performance  on  the  parent  system.  Except  for  a  few 
cases,  our  review  failed  to  reveal  effectiveness  data  based  on  operational  equipment. 
Although  performance  data  provide  the  most  valid  basis  for  determining  training  effective¬ 
ness,  such  data  are  sometimes  difficult  and  expensive  to  obtain.  Walch,  Yee,  and  Burright 
(1991)  have  described  the  dilemma  of  collecting  performance-based  effectiveness  data  as 
follows:  When  effectiveness  data  are  most  needed  (i.e.,  during  initial  system  develop¬ 
ment),  these  data  do  not  exist;  when  valid  performance  data  become  available  (i.e.,  after 
fielding),  users  no  longer  need  these  data  to  justify  the  system. 

Given  the  problems  with  obtaining  performance  data,  training  researchers  and 
developers  instead  have  used  a  variety  of  other  measures  that  relate  to  training 
effectiveness,  such  as  subjective  judgments  of  effectiveness,  assessments  of  user 
friendliness,  and  the  frequency  or  amount  of  usage.  Although  such  subjective  data  are 
related  to  aspects  of  training  effectiveness,  these  data  are  clearly  a  less-than-desirable 
substitute  for  objective  performance  data. 
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We  were  successful  in  finding  evidence  related  to  the  effectiveness  of  26  of  the  56 
systems  that  we  initially  identified,  although,  in  most  cases,  the  evidence  was  based  on 
subjective  appraisals  as  opposed  to  objective  performance.  (The  evidence  is  summarized  in 
the  part  of  Appendix  D  labeled  “Assessment”)  For  each  system  that  received  such  an 
assessment,  we  also  identified  features  that  some  have  identified  as  possibly  contributing  to 
(positive  features)  or  detracting  from  (negative  features)  the  effectiveness  of  the  ET  system. 
These  features  relate  directly  to  the  common  training  functions  of  ET  systems  identified  at 
the  beginning  of  this  document.  Table  3  presents  some  of  the  more  prevalent  positive  and 
negative  features  that  we  identified. 


Table  3.  Common  Positive  and  Negative  Features  on  ET  Systems 


Positive  Features 

Negative  Features 

Ability  to  author  custom  scenarios 

Access  to  library  of  "^cannecf  scenarios 

Ability  to  interface  with  other  related  systems  for  full- 
team  training 

Ability  to  mix  live  and  simulated  targets 

Provision  of  individualized  instruction  by  embedded 
tutors,  requiring  little  or  no  supervision 

Ability  to  emulate  system  inputs  realistically 

Problems  or  limitations  in  simulation  fidelity 

Reduction  in  or  interference  with  operational 
capabilities 

Lack  of  performance  measurement  capability 

Unacceptable  time  and  effort  required  to  switch 
between  operational  and  training  modes 

Lack  of  docurnentation  for  ET  system 

Increase  in  maintenance  and  logistics  requirements 

Of  the  26  ET  systems  for  which  we  had  effectiveness  information,  21  (81  percent) 
received  overall  favorable  assessments.  In  six  instances,  however,  the  favorable  assess¬ 
ments  were  qualified  because  users  noted  some  aspects  of  the  systems  that  they  felt 
detracted  from  training  effectiveness  or  because  positive  assessments  were  only  based  on 
preliminary  tests  of  the  system.  Favorably  assessed  systems  had  few  specific  positive 
features  in  common.  However,  they  shared  two  general  positive  characteristics:  they  were 
reliable  training  systems,  and  their  parent  systems  were  generally  available  for  training. 

For  the  unfavorably  assessed  systems,  no  single  feature  or  type  of  feature  stood 
out.  Interestingly,  negative  assessments  did  not  appear  to  be  related  to  the  system’s 
training  features.  For  instance,  the  Performance  Measurement  and  Evaluation  (PME)  for 
sonar  systems  of  surface  ships  received  an  unfavorable  assessment  even  though  it  is 
instructionally  sound.  Its  problem  was  an  inability  to  provide  training  without  other 
devices.  Instead,  negative  assessments  appeared  to  pertain  to  logistical  or  operational  con¬ 
siderations.  Two  examples  include  the  TPQ-29  Trainer  for  the  Improved  HAWK 
(IHAWK)  Missile  System,  which  is  especially  difficult  to  set  up  and  tear  down,  and  the 
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Operational  Training  Software  (OTS)  for  simulating  EW  targets  on  surface  ships,  which 
interferes  with  operation  of  the  AN/SLQ-32  console.  Thus,  the  negative  assessments  did 
not  indicate  a  disapproval  of  ET  concepts;  rather,  they  appeared  to  result  from  inadequate 
planning  and  implementation  of  ET  systems. 

3 .  Costs 

Two  similar  approaches  to  analysis  of  ET  costs  are  relevant.  Knapp  and  Orlansky 
(1983)  describe  a  generic  cost  element  structure  for  training  course,  procedure,  or  device. 
Their  major  cost  categories  include  costs  related  to  research  and  development  (R&D),  initial 
investment,  and  operation  and  support.  Individual  cost  elements  are  defined  within  these 
three  categories.  Witmer  and  Knerr  (1996)  suggested  a  similar  scheme  with  four  categories 
of  costs  related  to  design  and  development,  procurement,  maintenance,  and  operations. 
Both  approaches  provide  a  comprehensive  scheme  for  identifying  costs  related  to  the 
development  and  implementation  of  ET  systems. 

Cost  information  was  available  on  only  7  of  the  54  ET  systems  that  we  identified. 
Table  4  summarizes  this  information.  As  shown,  these  studies  considered  a  variety  of 
costs,  including  initial  cost,  operating  cost,  cost  comparisons,  and  cost  avoidances.  As 
shown  in  Table  4,  the  costs  examined  in  these  seven  studies  present  a  variety  of  methods 
and  details  concerning  costs.  Because  of  the  unique  nature  of  these  findings,  each  is 
described  below  as  a  case  study  in  cost  analysis.  Although  these  studies  examined 
different  types  of  costs,  they  were  consistent  in  their  suggestion  that  savings  that  may  result 
from  implementing  ET. 

a.  Advanced  Embedded  Training  (AET) 

AET,  an  R&D  project  for  the  Naval  Air  Warfare  Center  (NAWC),  is  part  of  an 
Advanced  Technology  Demonstration  (ATD)  to  integrate  human  performance  technologies 
in  the  AEGIS  Combat  System  and  Combat  Training  System.  The  eventual  product  will 
provide  an  adjunct  to  ACTS.  The  R&D  Project  Summaries  for  FY  1997  (NAWC,  1996) 
provided  the  following  estimated  cost  savings  but  did  not  include  costs  to  develop  and 
implement  the  AET. 

The  AET  system  will  significantly  reduce  the  requirements  for  assignment  of  Afloat 
Training  Organizations  and  AEGIS  Training  Support  Group  training  support  personnel  to 
ships,  which  could  result  in  a  $1.5M/year  cost  savings.  Moreover,  the  expected  level  of 
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Table  4.  Summary  of  Cost  Findings  Reiated  to  ET  Systems 


ET  System 

Costs  Considered 

Estimated  Savings 

Reference(8) 

AET 

Estimated  personnel 
costs  are  related  to 
training  effectiveness. 

Estimated  reduction  in 
training  support  person¬ 
nel  could  save  $1 .5M  per 
year,  and  increase  team 
performance  25  to 

40  percent. 

(Naval  Air  Warfare 

Center,  1996) 

Proposed  ET  System  for 
Emerging  Fighter  Aircraft 

Estimated  operating 
costs  are  related  to 
training  efficiency. 

3X  increase  in  training 
events  could  be 
achieved  for  1.2-1 .4X 
increase  in  costs. 

O’Brien  and  Hess 
(1988a,  1988b,  1988c, 
and1988d) 

OBEWS 

Comparison  of  initial 
investment  costs  of  ET 
system  to  parent 
system. 

Total  acquisition  cost  for 
OBEWS  {$121M)  is  less 
than  the  cost  of  a  single 
advanced  combat 

aircraft. 

Senate  Appropriations 
Committee  (1995) 

Onboard  Simulation 
(OBS)fortheF-15B 

Actual  cost  savings  In 
developing  system  soft¬ 
ware  and  operating 
equipment  during  opera¬ 
tional  test  of  parent  sys¬ 
tem  are  related  to  actual 
costs  to  develop  ET 
system. 

Savings  in  software 
development  costs  was 
$6.3M,  and  savings  in 
test  operation  was 
$351 K.  In  comparison, 
the  cost  of  OBS  devel¬ 
opment  was  $1.2M. 

Kocher  (1984) 

OCSTT 

Comparison  of  initial 
investment  costs  of 
similar  systems  is 
related  to  functionality. 

OCSTT,  costing  less 
than  $200K,  replaces 
Device  20B5,  costing 
$10M  per  unit.  The  20B5 
is  an  umbilical  pierside 

ET  system,  whereas  the 
OCSTT  is  an  onboard 
system  that  provides 
about  70  percent  of  the 
20B5  functions. 

Stratton  et  al.  (1996) 

SPA-25G  Embedded 
Training  System  (SETS) 

Avoidance  of  wasted 
training  costs. 

Based  on  an  estimated 
drop  in  attrition  in  Navy 
schools,  SETS  may 
save  $870K  per  year. 

Lacy,  Ellis,  and  Madden 
(1990) 

Tank  Weapons  Gunnery 
Simulation  System 
(TWGSS)  and  Precision 
Gunnery  System  (PGS) 

Avoidance  of  live-fire 
training  costs. 

Reductions  to  live 
ammunition  costs  saves 
about  $13M  annually  for 
each  system. 

B.  Harrison  (personal 
communication,  March 
1997) 
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training  enhancement  is  a  25  to  40  percent  step  improvement  in  team  performance,  as 
determined  by  validated  measures  of  effectiveness. 

b.  Embedded  Training  for  Emerging  Fighter  Aircraft 

Dynamics  Research  Corporation  produced  a  four-volume  document  for  the  Air 
Force  Aeronautical  Systems  Division  (ASD).  This  dociunent  describes  methods  for  deter¬ 
mining  ET  technology  in  emerging  systems  (O’Brien  and  Hess,  1988a,  1988b,  1988c,  and 
1988d).  One  of  the  emerging  systems  they  examined  was  the  F-16C,  a  (then)  near-term 
aircraft  scheduled  to  become  operational  in  the  1987-1993  timeframe.  A  central  component 
of  the  model  was  a  method  for  projecting  the  operational  costs  of  ET.  One  of  their  training 
models  assumes  that  pilots  mix  ET  and  live  training  events  (i.e.,  maneuvers)  during 
training  missions.  The  analysts  asserted  that  pilots  could  fit  in  twice  as  many  ET  events  as 
live  events  during  the  same  period  of  time  required  to  perform  the  live  events  alone.  In 
other  words,  O’Brien  and  Hess  contend  that  their  proposed  ET  training  system  would 
realize  savings  by  making  training  more  efficient. 

To  cost  out  these  events,  O’Brien  and  Hess  (1988c)  developed  two  models:  one 
for  training  without  ET  and  one  for  training  with  ET.  The  major  cost  elements  for  training 
without  ET  included  aircraft  flying  hours,  gun  rounds,  bombs,  and  missiles.  The  elements 
for  training  with  ET  included  flying  hours  but  did  not  include  gun  rounds,  bombs,  and 
missiles,  and  it  added  costs  for  “wear  and  tear”  on  avionics,  ground  planning  and 
debriefing,  software,  and  ET  operation.  Analysis  of  the  previous  “mixed  scenario” 
showed  that  a  large  increase  in  events  (in  effect,  tripling  the  number  of  trials)  could  be 
achieved  by  a  relatively  modest  increase  in  total  costs  (20  to  40  percent).  This  analysis 
implies  that  the  proposed  ET  system  would  be  cost  effective  if  simulated  trials  provide  at 
least  20  to  40  percent  of  the  benefit  of  live  trials — a.  modest  expectation  for  the  effectiveness 
of  the  system. 

c.  On-Board  Electronic  Warfare  Simulator 

Some  data  exist  on  system  acquisition  costs  for  the  OBEWS.  During  their  delib¬ 
erations  on  the  FY  1995  budget,  the  Senate  Appropriations  Committee  (SAC)  compared  the 
costs  of  acquiring  OBEWS  to  the  costs  of  acquiring  its  parent  system: 

The  Committee  notes  that  the  total  acquisition  cost  for  OBEWS  was  esti¬ 
mated  to  be  $120,600,000  for  approximately  900  aircraft  systems  and 

40  squadron  debriefing  stations.  That  amount  is  less  than  the  probable  cost 

of  a  single  advanced  combat  aircraft  in  the  future  (SAC,  1995,  p.  287). 
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d.  Onboard  Simulation  for  the  F>15B 


The  F-15B  OBS  is  a  fully  embedded  training  system  that  presents  simulated  targets 
in  the  pilot’s  heads-up  display  (HUD).  Kocher  (1984)  documented  the  costs  saved  during 
operational  testing  of  the  Integrated  Flight  and  Fire  Control  (IFFC)  system  on  the  F-15B. 
The  original  concept  of  the  OBS  was  to  serve  as  a  replacement  for  and  improvement  on  a 
ground-based  test  unit  that  would  send  signals  to  the  aircraft  subsystems.  During  IFFC 
development,  the  OBS  was  used  as  a  man-in-the-loop  simulation  to  develop  and  debug 
software.  Kocher  figured  that  using  OBS  reduced  the  man-hours  per  computer  word  from 
8  to  3.5.  This  resulted  in  a  savings  of  about  $7.5M.  Subtracting  the  cost  to  develop  OBS 
($1.2M),  the  total  savings  in  software  development  amounted  to  $6.3M. 

OBS  also  reduced  the  operational  costs  of  IFFC  flight  testing  in  two  ways. 

1.  OBS  reduced  expenses  related  to  using  real  targets.  These  expenses  included 
the  cost  of  ammunition  and  bombs  and  cost  per  hour  to  operate  additional  air¬ 
craft  to  provide  or  tow  targets. 

2.  OBS  greatly  increased  the  number  of  events  that  can  be  trained  per  flight  hour 
because  of  its  ability  to  reset  conditions  instantaneously  after  a  simulated 
weapon  encounter  to  prepare  for  the  next  encounter. 

In  comparison,  lengthy  set-up  times  are  required  between  encounters  using  real  targets.  On 
average,  OBS  provided  39.8  events  per  hour,  whereas  tactical  (nonfiring)  training  averaged 
16.3  events  and  live-fire  training  averaged  5.3  events.  Total  savings  were  calculated 
assuming  that  actual  OBS  events  were  replaced  by  tactical  events.  The  actual  savings  was 
about  $351K. 

Finally,  the  total  savings  attributable  to  OBS  (about  $6.6M,  including  software 
development  and  operational  costs)  were  compared  to  the  cost  to  develop  OBS  ($1.2M). 
In  other  words,  the  savings  realized  were  5.5  times  the  costs  required  to  develop  the  ET 
system. 


e.  Organic  Combat  System  Training  Technology 

NAWC’s  Training  Technical  Division  has  developed  the  OCSTT  as  a  technology 
demonstration  for  showing  how  modem  off-the-shelf  digital  processing  technology  can  be 
used  to  provide  combat  training  capability  aboard  ship  (Stratton  et  al.,  1996).  OCSTT 
represents  a  modernization  of  Device  20B5,  which  is  an  umbilical  pierside  system  for 
training  combat  teams  on  FFG-7  class  surface  ships.  The  OCSTT  provides  the  capability 
to  train  about  70  percent  of  the  functions  trained  by  Device  20B5  at  sea.  The  redesigned 
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system  is  also  compatible  with  DIS  standards  so  that  it  can  be  incorporated  into  the  Battle 
Force  Tactical  Trainer  (BFTT),  a  Navy  project  for  linking  different  training  systems  via 
wide  area  networking.  Moreover,  the  OCSTT  provides  these  training  capabilities  at  sub¬ 
stantially  lower  investment  costs:  a  single  production  20B5  costs  in  excess  of  $10M, 
whereas  the  OCSTT  costs  less  than  $200K.  This  provides  a  demonstration  of  how  modem 
digital  technology  is  greatly  decreasing  the  costs  of  ET. 

f.  SPA-25G  Embedded  Training  System 

SETS  trains  individual  operator  tasks  in  four  task  areas:  equipment  proficiency,  radar 
navigation,  air  intercept  control,  and  anti-submarine  air  control.  According  to  Lacy,  Ellis, 
and  Madden  (1990),  instructors  in  the  Navy  “C”  school  for  air  controllers  estimate  that 
SETS  could  reduce  attrition  by  50  percent.  The  annual  throughput  at  the  school  is  58  stu¬ 
dents,  and  training  costs  approximately  $30K  per  student.  Thus,  using  SETS  avoids 
$870K  per  year  (29  x  $30K)  in  potentially  wasted  training  funds. 

g .  Tank  Weapons  Gunnery  Simulation  System  and  Precision 
Gunnery  System  (TWGSS) 

The  Army’s  Simulation  Training  and  Instramentation  Command  (STRICOM)  has 
provided  cost  data  related  to  the  TWGSS  and  the  PGS  (B.  Harrison,  personal  communica¬ 
tion,  March  1997).  The  TWGSS  is  an  appended  ET  system  for  the  M  1-series  batde  tank. 
TWGSS  uses  a  laser  emission  system  to  simulate  the  effects  of  precision  firing  of  the 
120-mm  main  gim  and  coaxial  machine  gun  for  force-on-force  (FOF)  tactical  training.  The 
PGS  is  an  analogous  system  for  the  Bradley  Infantry  Fighting  Vehicle  and  simulates  the 
effects  of  the  25-mm  main  gun,  coaxial  machine  gun,  and  the  Tube-launched,  Optically- 
tracked,  Wire-guided  (TOW)  missile. 

The  basis  of  issue  (BOI)  for  TWGSS  is  one  per  platoon  (four  tanks).  The  armual 
allocation  of  practice  ammunition  for  each  tank  crew  in  a  platoon  receiving  TWGSS  is 
reduced  by  10  rounds.  Main  gun  rounds  cost  approximately  $600  each.  Assuming  that 
2,166  active  duty  tanks  are  in  the  inventory,  the  Army  avoids  nearly  $13  million  in  main 
gun  ammunition  costs.  This  analysis  does  not  include  the  costs  and  savings  of  ammunition 
for  the  coaxial  machine  gun  nor  does  it  include  the  costs  of  buying  and  using  TWGSS. 

A  similar  analysis  can  be  performed  on  PGS.  Each  Bradley  crew  is  getting  cut 
192  rounds  at  $30  each.  Given  approximately  2,200  Bradleys  in  use,  the  cost  avoidance 
related  to  main  gun  ammunition  is  $12.7  million.  This  does  not  include  potential  avoidance 
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of  coaxial  machine  gun  ammunition  and  TOW  missiles  costs.  Potential  cost  avoidance  in 
TOW  missiles  is  great,  given  that  TOWs  cost  approximately  $12,000  each.  However, 
crews  fire  very  few  of  these  missUes.  As  in  TWGSS,  this  analysis  overlooks  the  costs  of 
buying  and  using  PGS. 

4.  Research  on  Related  Technologies 

There  is  a  dearth  of  empirical  research  on  the  effectiveness  and  costs  related  to  ET; 
however,  there  is  a  substantial  body  of  research  on  stand-alone,  computer-based  instruction 
(CBI)  and  simulator  training.  Because  the  training  functions  and  technologies  are  common 
to  stand-alone  and  ET  methods,  this  empirical  research  base  is  relevant  to  the  evaluation  of 
ET.  In  that  regard,  we  offer  the  following  four  summary  points,  which  are  based  on  over 
100  studies  of  computer-based  training  systems. 

1.  CBI  is  as  effective  as  classroom  instruction  (i.e.,  students  learn  as  much)  and 
saves  about  30  percent  of  the  time  needed  for  conventional  instruction.  The 
main  reason  for  this  effect  is  that  students  proceed  at  their  own  rates  of 
learning  and  that  most  of  the  students  are  not  restrained  by  the  slower  learning 
rates  of  some  students  (Orlansky  and  String,  1979). 

2.  CBI  improves  performance  so  that  the  average  student  performs  like  those  at 
the  69“'  percentile  of  those  using  conventional  instruction  (Hetcher,  1990). 

3.  Pilots  who  use  flight  simulators  for  training  initial  flying  tasks  save  about  half 
the  flight  time  needed  to  train  the  same  tasks  in  an  aircraft  Tlie  cost  per  hour 
of  using  a  simulator  for  training  is  about  10  percent  of  what  it  costs  to  use  an 
equivalent  airplane.  Thus,  the  use  of  flight  simulators  for  training  is  highly 
cost-effective  and  enhances  training  in  the  aircraft  (Orlansky  et  al.,  1994). 

4.  The  use  of  computer-based  guidance  for  maintaining  F-16  aircraft,  compared 
to  the  use  of  conventional  paper-based  technical  orders,  increased  the  percent 
of  maintenance  problems  solved  successfully,  reduced  the  time  needed  to 
complete  maintenance  actions  and  order  spare  parts,  and  reduced  the  number 
of  errors  per  problem  for  specialists  and  nonspecialists.  In  fact,  technicians 
using  computer-based  guides  performed  better  than  specialists  using  paper- 
based  technical  orders.  The  use  of  computer-based  maintenance  for  the  F-16 
was  estimated  to  save  about  $21.6M  per  year  (FY  1995  dollars),  about  0.5 
percent  of  the  annual  budget  for  operations  and  maintenance  of  the  F-16  alone 
(Teitelbaum  and  Orlansky,  1996). 
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F.  FINDINGS 


The  purpose  of  ET  is  to  provide  effective  training  to  operational  units  and  indi¬ 
viduals  using  operational  equipment  ET  requires  the  ability  to  provide  relevant  threats  and 
scenarios,  assessment  of  performance,  feedback  for  lessons  learned,  and  collection  and 
management  of  data  on  unit  training.  The  “embedded”  aspect  concerns  the  incorporation  of 
a  training  capability  in  the  operational  equipment  Die  choice  of  whether  to  embed  training 
must  be  evaluated  on  a  case-by-case  basis,  depending  on  the  feasibility  and  costs  of  incor¬ 
porating  training  in  the  operational  equipment 

The  main  advantage  of  ET  is  the  ability  to  train  with  the  same  operational  equipment 
that  the  unit  takes  to  war.  Training  can  take  place  at  home  station  or  while  deployed.  This 
type  of  training  has  high  fidelity,  is  concurrent  with  the  latest  upgrades  to  the  equipment, 
can  be  linked  to  large-scale  exercises  as  the  capability  of  the  Defense  Simulation  Internet 
(DSI)  grows,  and  permits  mission  rehearsal  and  assessment  of  force  readiness.  Properly 
analyzed  and  configured,  ET  systems  are  intended  to  enhance  the  use  and  effectiveness  of 
the  operational  hardware. 

The  disadvantage  of  ET  is  that  it  may  reduce  the  capability  of  operational  equipment 
by  adding  weight,  reducing  space,  and  increasing  wear  and  maintenance.  Furthermore, 
when  ET  functions  are  being  used,  the  capabilities  of  the  operational  equipment  may 
become  compromised  or  unavailable.  However,  many  of  the  reported  disadvantages  are 
often  the  result  of  inadequate  planning  and  implementation  and  are  not  the  necessary  result 
of  an  effort  to  embed  training. 

According  to  user  reports  identified  in  our  review  of  the  literature,  ET  is  effective 
and  generally  acceptable.  Nevertheless,  little  valid  and  reliable  empirical  data  exist  on  ET’s 
cost  and  effectiveness.  An  effort  to  collect  this  information  is  needed. 

Army  and  Navy  Service  guidance  has  promoted  ET  as  the  preferred  training  alter¬ 
native  for  developing  systems.  Defense  policy  should  continue  to  favor  ET  as  a  require¬ 
ment  in  new  weapon  systems  until  it  is  ruled  out  in  any  particular  case  by  careful  review  of 
its  probable  effectiveness  and  cost.  Such  consideration  from  the  start  of  new  programs 
offers  the  best  possibility  of  training  that  will  optimize  the  performance  and  effectiveness  of 
new  weapons. 
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GLOSSARY 


2-D 

two  dimensional 

3-D 

three  dimensional 

AAR 

After- Action  Review 

AAWC 

anti-aircraft  warfare  coordinator 

ABCS 

Army  Battle  Command  System 

ACE 

Air  Combat  Evaluator 

ACTER 

Anti-Countermeasures  Trainer 

ACTS 

AEGIS  Combat  Training  System 

AD 

air  defense 

ADA 

Air  Defense  ArtiUery 

ADDC 

Air  Defense  Direction  Center 

ADSIM 

Automatic  Detection  Tracking  System  Simulation 

ADT 

Automatic  Detector  Tracker 

AET 

Advanced  Embedded  Training 

AFATDS 

Advanced  Field  Artillery  Tactical  Data  System 

AFCS 

Automated  Fire  Control  System 

AHST 

Abrams  Full  Crew  Interactive  Simulator  Trainer 

AGES 

Air  Groimd  Engagement  System 

AIMS 

Automated  Information  Management  System 

AMRAAM 

advanced  medium  range  air-to-air  missile 

AOCP 

Airborne  Operational  Computer  Program 
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ARI 

Army  Research  Institute 

ARI 

Army  Research  Institute 

ARTEP 

Army  Training  and  Evaluation  Plan 

ASAS 

All  Source  Analysis  System 

ASD 

Aeronautical  Systems  Division 

ASE 

Aircraft  Survivability  Equipment 

ASET 

Aircraft  Survivability  Equipment  Trainer 

ASG 

Advanced  Scenario  Generator 

ASW 

anti-submarine  warfare 

ATAS 

Advanced  Tank  Armament  System 

ATCSS 

Army  Tactical  Command  and  Control  System 

AID 

Advanced  Technology  Demonstration 

AWACS 

Airborne  Warning  and  Control  System 

AWS 

anti-submarine  warfare 

BATR 

BuUets-At-Target-Range 

BFACS 

Battlefield  Functional  Area  Command  and  Control  Systems 

BFTT 

Battle  Force  Tactical  Trainer 

BIN 

BFTT  Information  Net 

BIT 

built-in  test 

BOI 

basis  of  issue 

C&D 

command  and  decision 

C/NET 

Chief  of  Navy/Naval  Education  and  Training 

C2 

command  and  control 

C3I 

command,  control,  communications,  and  intelligence 

C4I 

command,  control,  communications,  computers,  and 
intelligence 
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CAI 

computer-assisted  instruction 

CAST 

Computer-Aided  Submode  Training 

CBI 

computer-based  instruction 

CBT 

computer-based  training 

CD 

compact  disk 

CIC 

Combat  Information  Center 

CINC 

Commander  in  Chief 

CINCLANTFLT 

Commander  in  Chief,  Atlantic  Reet 

CINCPACFLT 

Commander  in  Chief,  Pacific  Fleet 

CJTF 

Combined  Joint  Task  Force 

CNO 

Chief  of  Naval  Operations 

CO 

Commanding  Officer 

COACH 

Computerized  On-line  Advising  and  Contextual  Help 

COE 

common  operating  environment 

COTS 

commercial  off-the-shelf 

CP 

capabilities  package 

CSC 

combat  systems  coordinator 

CSS 

combat  service  support 

CSSCS 

Combat  Service  Support  Command  System 

CSTS 

Combat  Simulation  Training  [formerly,  Test]  System 

CTC 

Combat  Training  Center 

D-FIRST 

Deployable  Force-on-Force  Instrumented  Range  SysTem 

DIS 

Distributed  Interactive  Simulation 

DoD 

Department  of  Defense 

DoDHS 

DoD  Intelligence  Information  System 

DOS 

Disk  Operating  System 
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DSI 

Defense  Simulation  Internet 

DTIC 

Defense  Technical  Information  Center 

ECCM 

electronic  counter-countermeasure 

ECM 

electronic  countermeasure 

EEPROM 

electrically  erasable,  programmable  read  only  memory 

ENSCE 

Enemy  Situation  Correlation  Element 

EO 

Executive  Officer 

ESM 

electronic  support  measure 

ET 

embedded  training 

EW 

electronic  warfare 

EWIET 

Electronic  Warfare  Intelligent  Embedded  Training 

EWS 

electronic  warfare  supervisor 

FAADC2I 

Forward  Area  Air  Defense,  Command,  Control,  and 
Intelligence 

PCS 

Fire  Control  System 

FM 

field  manual 

FY 

Fiscal  Year 

GFCS 

Gun  Fire  Control  System 

GPS 

Global  Positioning  System 

GSS 

ground  support  subsystem 

HAWK 

Homing  AU-the-Way  Killer 

HIP 

Howitzer  Improvement  Program 

HIPET 

Howitzer  Improvement  Program  Embedded  Training 

HUD 

heads-up  display 

I-TS 

Improved  Target  Acquisition  System-Training  System 

ICW 

Interactive  Courseware 
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IDC 

identification  coordinator 

lEOA 

Intelligent  Embedded  Operator  Assistant 

IFCC 

Integrated  Flight  and  Fire  Control 

IFF 

identification  friend  or  foe 

IFT 

In-Flight  Trainer 

IHAWK 

Improved  HAWK 

IMAT 

Interactive  Multisensor  Analysis  Training 

IMIS 

Integrated  Maintenance  Information  System 

INDEX 

Independent  Exercises 

lOT 

Integral  Operator  Trainer 

UAS 

Improved  Target  Acquisition  System 

Janus 

large  combat  model  (constructive  simulation) 

JCMT 

Joint  Collection  Management  Tools 

JCPMS 

Joint  Communications  Planning  and  Management  System 

JMCIS 

Joint  Maritime  Command  Information  System 

JSIMS 

Joint  Simulation  Systems 

JTF 

Joint  Task  Force 

KOALAS 

Knowledgeable  Observation  Analysis-Linked  Advisory 
System 

L-mAN 

Lesson  Translator 

LAMPS 

Light  Airborne  Multi-Purpose  System 

LAT 

Live  Air  Trainer 

MCnST 

Marine  Corps  Full-Crew  Interactive  Simulation  Trainer 

METT-T 

mission,  enemy,  troops,  terrain,  and  time 

MFCS 

Missile  Fire  Control  System 

MILES 

Multiple  Integrated  Laser  Engagement  System 
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NAWC 

Naval  Air  Warfare  Center 

NRL 

Naval  Research  Laboratory 

NTCS 

Navy  Tactical  Command  System 

NTDS 

Navy  Tactical  Data  System 

QC 

observer/controller 

OBEWS 

On-Board  Electronic  Warfare  Simulator 

OBS 

Onboard  Simulation 

on-board  subsystem 

OCSTT 

Organic  Combat  System  Training  Technology 

OPNAVINST 

Operational  Navy  Instruction 

ORATS 

Operational  Readiness  Assessment  and  Training  System 

OTS 

Operational  Training  Software 

PAL 

Presentation  Authoring  Language 

PARS 

Patrol  Analysis  Recording  System 

PC 

personal  computer 

PGS 

Precision  Guruiery  System 

PMA 

Portable  Maintenance  Aid 

PME 

Performance  Measurement  and  Evaluation 

PPI 

Plan  Position  Indicator 

R&D 

research  and  development 

RADDS 

Radar  Display  and  Distribution  System 

RAM 

random  access  memory 

reliability,  availability,  and  maintenance 

RCS 

Radio  Communications  System 

REES 

Radar  Electromagnetic  Environmental  Simulator 

RESS 

Radar  Environment  Simulator  System 

RIO 

radar  intercept  officer 
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ROM 

read  only  memory 

RPV 

Remotely  Piloted  Vehicle 

RWR 

radar  warning  receiver 

S&T 

Science  and  Technology 

SAC 

Senate  Appropriations  Committee 

SAGE 

Semi-Automatic  Ground  Environment 

SAS 

Surveillance  and  AIMS  System 

SDC 

situation  display  console 

SETS 

SPA-25G  Embedded  Training  System 

SOR 

State  of  Readiness 

ST 

Sonar  Technician 

STAMIS 

Standard  Army  Management  Information  Systems 

STOW 

Synthetic  Theater  of  War 

STP 

Systems  Training  Program 

STRICOM 

Simulation  Training  and  Instrumentation  Command 

STSS 

Sonar  Target  Signal  Simulator 

STX 

Situational  Training  Exercise 

TAC 

Training  Alarm  Controller 

Tactical  Air  Command 

TACFIRE 

Tactical  Fire  Direction  System 

TADSS 

training  aids,  devices,  simulations,  and  simulators 

TAO 

tactical  action  officer 

TCG 

Training  Coordination  Group 

TES 

Tactical  Engagement  Simulation 

TI 

Texas  Instruments 

TEC 

tactical  information  coordinator 

GL-7 


TIU 

Training  Interface  Unit 

IM 

technical  manual 

TOW 

Tube-launched,  Optically-tracked,  Wire-guided  missile 

TFT 

Troop  Proficiency  Trainer 

TRADOC 

Training  and  Doctrine  Command 

TRASANA 

TRADOC  Systems  Analysis  Activity 

TT&P 

tactics,  techniques,  and  procedures 

TWGSS 

Tank  Weapons  Gunnery  Simulation  System 

UFCS 

Underwater  Fire  Control  System 

VSS 

Video  Signal  Simulator 

WCS 

Weapons  Control  System 

WDS 

Weapons  Direction  System 

WHO 

Weapon  Engineer  Officer 

WF 

Warfighter 

WSRT 

Weapon  Systems  Readiness  Test 

WWMCCS 

World-Wide  Military  Command  and  Control  System 
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APPENDIX  A. 

TYPES  OF  EMBEDDED  TRAINING  SYSTEMS, 
WITH  EXAMPLES 


Our  definition  of  ET  includes  many  different  types  of  technologies,  each  with 
unique  aspects  of  effectiveness  and  cost.  TTie  following  is  an  attempt  to  identify  different 
types  of  ET  systems  and  their  associated  features.  Its  purpose  is  not  to  provide  a  mutually 
exclusive  classification  scheme  but,  rather,  to  identify  the  various  technologies  that  can 
provide  ET  functions.  To  make  these  categories  more  concrete,  we  have  included  some 
examples, 

A.  FULLY  EMBEDDED  SYSTEMS 

The  prototype  ET  system,  called  “fully  embedded,”  has  all  of  its  training  functions 
built  into  the  operational  weapon  system.  The  ideal,  fully  embedded  training  system  pro¬ 
vides  high-fidelity  simulated  scenarios  along  with  automated  training  support,  including 
performance  measurement  and  feedback  capabilities.  Although  this  is  the  most  comprehen¬ 
sive  form  of  ET,  it  is  usually  limited  in  scope  to  the  most  highly  critical  aspects  of  combat 
engagements.  No  ET  system  can  or  pretends  to  provide  all  necessary  system  training. 
Functionally,  fully  embedded  training  systems  are  not  different  from  training  simulators, 
which  have  been  shown  to  be  cost-effective  means  of  delivering  training  (Orlansky  et  al., 
1994). 

Although  the  fully  embedded  form  is  the  most  often  described  and  almost  univer¬ 
sally  endorsed  type  of  ET,  there  are  surprisingly  few  fielded  examples  of  this  category: 

•  The  Troop  Proficiency  Trainer  (TPT).  The  TPT  is  embedded  in  the 
Army’s  PATRIOT  Missile  System.  The  TPT  simulates  actual  PATRIOT  data 
while  disabling  the  radar  system.  Each  battery  can  train  individually  or  be 
networked  to  simulate  battalion-directed  air  defense  functions. 

•  The  Airborne  Operational  Computer  Program  (AOCP).  The  AOCP 
is  embedded  in  the  Air  Force’s  Airborne  Warning  and  Control  System 
(AW ACS).  The  AOCP  presents  simulated  scenarios  on-  or  off-line.  If  on¬ 
line,  the  ET  is  automatically  deactivated  when  AW  ACS  receives  an  alert. 
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•  The  AEGIS  Combat  Training  System  (ACTS).  ACTS  is  embedded  in 
the  Navy’s  AEGIS  Weapon  System.  ACTS  uses  simulation  scenarios  to  train 
command  and  decision-making.  Weapons  Control  System  (WCS)  functions, 
and  Fire  Control  System  (FCS)  functions.  ACTS  trains  all  Combat  Informa¬ 
tion  Center  (CIC)  stations  and  can  be  used  for  multi-ship  training  by 
transmitting  computer-generated  symbols  and  raw  video  to  other  participants. 

•  The  Battle  Force  Tactical  Trainer  (BFTT).  The  BFTT  provides  tacti¬ 
cal  training,  on  a  part  basis,  by  linking  fully  embedded  systems  of  surface, 
subsurface,  and  air  forces  for  a  world-wide  synthetic  theater  of  war  (STOW). 
Training  capability  includes  timely  feedback.  This  system  is  currently  under 
development 

B .  STRAP-ON  (OR  APPENDED)  SYSTEMS 

Strap-on  ET  systems  use  hardware  that  is  added  or  appended  to  the  operational 
equipment  to  make  it  more  useful  for  training.  There  are  two  types  of  strap-on  systems, 
which  differ  in  whether  they  are  incorporated  into  the  initial  design  of  the  equipment  or  are 
retrofitted  after  the  equipment  is  designed  and  implemented. 

1 .  Integrated  Strap-On  Systems 

This  type  of  system  refers  to  appended  systems  that  are  planned  for  and  integrated 
into  the  operational  equipment.  Even  though  the  ET  system  is  external  to  the  equipment, 
this  form  of  strap-on  ET  is  a  planned  component  of  the  equipment.  This  sort  of  strap-on 
system  does  not  differ  functionally  from  fully  integrated  ET  systems.  The  following 
examples  of  this  type  are  provided: 

•  The  Tank  Weapons  Gunnery  Simulation  System  (TWGSS).  The 
TWGSS  was  designed  to  be  ported  into  the  FCS  of  the  M1A2  tank.  This  par¬ 
ticular  example  is  sometimes  used  as  an  example  of  an  “umbilical”  ET  system 
because  it  has  physical  connections  that  are  embedded  in  the  operational 
equipment. 

•  The  Training  Interface  Unit  (TIU).  The  TIU  for  the  Aquila  Remotely 
Piloted  Vehicle  (RPV)  straps  onto  the  system  to  provide  the  additional  com¬ 
puting  capacity  needed  to  run  a  computer-generated  simulation  through  the 
Aquila  system  displays. 
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2.  Post  Hoc  Strap-On  Systems 

Post  hoc  strap-on  ET  systems  are  those  systems  that  are  developed  independently 
from  and  often  after  the  design  of  the  operational  equipment.  This  category  includes  ET 
systems  that  have  been  retrofitted  to  existing  operational  equipment.  The  following  exam¬ 
ples  are  pertinent: 

•  The  On-Board  Electronic  Warfare  Simulator  (OBEWS).  OBEWS 
was  designed  to  simulate  electronic  warfare  (EW)  targets  in  F-16s.  The  strap- 
on  component  is  a  “pod”  mounted  under  the  wing.  It  interacts  with  the  air¬ 
craft’s  radar  system  to  produce  appropriate  signals. 

•  The  Multiple  Integrated  Laser  Engagement  System  (MILES). 
MILES  is  used  on  armored  vehicles  but  does  not  physically  interact  with  the 
PCS  of  the  tank.  It  is  embedded  only  in  the  sense  that  MILES  is  a  system  that 
employs  the  operational  system  to  provide  training. 

C.  PRACTICE  DEVICES 

IdeaDy,  ET  systems  should  provide  performance  assessment  and  training  manage¬ 
ment  capabilities  in  addition  to  the  opportunity  to  practice.  Practice  devices  refer  to  ET 
systems  that  provide  practice  to  the  exclusion  of  other  training  functions.  The  following 
examples  are  ET  strap-on  systems  that  provide  only  practice  capabilities.  Note,  however, 
that  these  systems  are  normally  supplemented  with  others  to  compensate  for  their  limited 
capabilities. 

•  MILES.  MILES  is  designed  to  provide  practice  in  force-on-force  (FOP) 
engagements.  It  does  not  provide  instruction  on  tactics,  performance  meas¬ 
urement  capabilities,  or  feedback.  MILES  is  often  used  in  the  context  of  a 
Situational  Training  Exercise  (STX)  to  provide  the  battle  scenario  context  and 
performance  scoring  and  feedback  procedures  for  observer/controllers  (0/Cs). 
STXs  are  part  of  the  text-based  Army  Training  and  Evaluation  Plan  (ARTEP). 

•  Unit  34.  Unit  34  generates  sonar  targets  on  AN/SQS-53  sonar  consoles  and 
Navy  Tactical  Data  System  (NTDS)  consoles  on  surface  ships.  Unit  34  is 
sometimes  used  with  the  Operational  Readiness  Assessment  and  Training 
System  (ORATS)  structured  training  package,  which  provides  the  missing 
instructional  components  to  this  ET  system. 

D.  EMBEDDED  COMPUTER-BASED  TUTORIALS 

Some  types  of  operational  equipment  employ  general-purpose  digital  computer 
technology  to  support  complex  decisions  related  to  command  and  control  (C2)  and  fire 
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direction.  Such  systems  are  usually  capable  of  running  standard  computer-based  tutorials, 
which  could  be  authored  using  conventional  off-the-shelf  software.  Functionally,  these 
tutorials  are  not  different  from  conventional  computer-based  training  (CBT),  a  technology 
demonstrated  to  be  cost-effective  (Orlansky  and  String,  1979;  Fletcher  and  Orlansky, 
1989).  This  approach  can  be  particularly  cost-effective  if  the  operating  system  of  the 
computer-based  system  is  compatible  with  CBT  authoring  systems.  In  that  case,  the  cost 
of  developing  new  training  programs  would  be  low.  Examples  of  computer-based  tutors 
include  the  following: 

•  The  prototype  tutorial  system  for  the  Air  Force’s  Sentinel  Byte. 

The  Sentinel  Byte  is  a  command,  control,  communications,  and  intelligence 
(C3I)  system  for  planning  air  routes  and  analyzing  air  defense  threats. 

•  The  computer«based  tutorials  embedded  in  the  older  Joint  Serv¬ 
ices  World-Wide  Military  Command  and  Control  System 
(WWMCCS).  A  unique  capability  of  this  system  is  that  tutorials  can  run  at 
some  stations  while  the  system  is  operating  on  others. 

•  The  computer-based  tutorials  in  the  Tactical  Fire  Direction  Sys¬ 
tem  (TACFIRE).  These  tutorials  were  developed  by  the  Army  Research 
Institute  (ARI)  in  the  PLANTT  authoring  language. 

E.  EMBEDDED  JOB  OR  PERFORMANCE  AIDS 

Job  or  performance  aiding  is  considered  to  be  a  separate  topic  from  training. 
However,  when  job  aids  for  complex  cognitive  tasks  (often  called  “decision  aids”)  are 
embedded  in  operational  equipment,  these  job  aids  closely  resemble  ET  systems.  Zachary 
and  Hicinbotham  (1994)  contend  that  the  major  similarities  of  ET  and  embedded  decision 
aids  are  that  both  of  them  focus  on  human  performance  improvement,  are  designed  from 
human  front-end  analyses,  and  use  similar  technology  for  development. 

On-line  helps  are  simple  (even  trivial)  examples  of  embedded  job  aids  and  are 
implemented  on  several  embedded  tutorials,  particularly  for  maintenance  tasks.  These 
types  of  aids  resemble  those  implemented  on  commercial  software.  Perhaps  more  inter¬ 
esting  are  embedded  performance  aids  for  complex  cognitive  tasks.  However,  these 
technologies  are  only  at  the  proposal  stage  of  development.  The  following  are  some  hypo¬ 
thetical  examples,  which  represent  straightforward  extensions  of  extant  technology. 


A-6 


1 .  On-line  Coaching 

It  has  been  proposed  that  maintenance  test  equipment  coach  the  performer  through 
troubleshooting  steps  as  he  performs  the  actual  test.  The  proposed  system  called  Comput¬ 
erized  On-line  Advising  and  Contextual  Help  (COACH)  appears  to  provide  some  of  these 
functions  as  an  embedded  feature  of  the  Portable  Maintenance  Aid  (PMA).  The  PMA  is 
small  ruggedized  component  of  the  Integrated  Maintenance  Information  System  (IMIS)  that 
is  designed  to  aid  in  flightline  maintenance.  COACH  embeds  an  advisor  (the  “Coach”)  in 
the  PMA  to  help  the  technician  solve  simulated  problems  in  training.  While  COACH  can 
be  used  in  the  flightline  maintenance  environment,  the  developers  caution  against  using  it 
during  actual  maintenance  operations  (Wilson,  Walsh,  Arnold,  and  Daly,  1996).  Ironi¬ 
cally,  then,  COACH  may  be  a  better  trainer  than  a  job  aid. 

2 .  Decision  Aiding 

It  has  also  been  proposed  that  C3I  information  management  systems  incorporate 
complex  decision  aids — even  including  the  capability  to  wargame  actions  during  prepara¬ 
tion  for  a  battle.  This  function  is  projected  for  the  Interactive  Multisensor  Analysis 
Training  (IMAT)  simulation  of  anti-submarine  and  EW  effects.  However,  development  of 
this  capability  assumes  that  the  present  land-based  demonstration  will  be  converted  to  an 
operational  on-board  (i.e.,  embedded)  system. 

3 .  Rehearsal 

Taken  a  step  further,  constructive  simulations  [e.g.,  Janus,  Joint  Simulation 
Systems  (JSIMS)]  could  be  incorporated  into  C3I  systems  to  simulate  battlefield  effects. 
Such  a  system  could  provide  battle  staffs  the  capability  to  rehearse  planned  actions  during 
preparation  for  a  battle. 

F.  CERTAIN  STAND-ALONE  SYSTEMS 

Stand-alone  systems  are  usually  considered  alternatives  to — rather  than  examples 
of — ^ET  systems.  However,  a  few  specialized  types  of  stand-alone  training  systems  are 
closely  related  to  the  concept  of  ET. 

1 .  Shipboard  Training  Systems 

The  Navy  considers  almost  any  training  system  mounted  on  ships  as  providing  ET. 
This  is  consistent  with  guidance  from  the  Chief  of  Naval  Operations  (1991)  as  cited  by  the 
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Naval  Surface  Warfare  Center  (1996)  that  .  .  the  ship,  when  properly  supported, 
presents  the  most  effective  training  site  for  appropriate  operational  and  functional  training.” 
This  would  include  shipboard  computer-based  tutors,  training  devices,  and  constructive 
simulations.  Compared  to  other  military  systems,  there  is  less  penalty  to  pay  for  the  weight 
and  space  of  a  stand-alone  training  system  on  a  ship,  particularly  one  that  is  pierside. 

2.  Distributed  Interactive  Simulation  (DIS)  Systems 

Certain  DIS  systems  are  related  to  ET  to  the  extent  that  they  allow  or  promote 
training  with  the  operational  equipment.  An  example  of  such  a  DIS-based  system  is  the 
Army  National  Guard’s  Deployable  Force-on-Force  Instrumented  Range  SysTem 
(D-FIRST).  D-FIRST  is  an  instrumented  range  system,  which  is  based  on  existing  Global 
Positioning  System  (GPS)  and  MILES  technology.  This  DIS-compatible  system  has  the 
capability  to  link  live  with  virtual  and  constructive  simulations. 

3.  Systems  Intended  To  Be  Embedded 

This  category  of  stand-alone  systems  include  those  systems  that  are  intended  to  be 
embedded  but  which,  for  cost  or  other  reasons,  have  not  been  implemented.  One  example 
is  the  Electronic  Warfare  Intelligent  Embedded  Training  (EWIET)  system  (Gluckman  and 
Willis,  1994).  This  is  a  proof-of-principle  demonstration  of  ET  technology  for  the  Navy’s 
SLQ-32  EW  system.  The  principle  being  demonstrated  is  the  application  of  intelligent 
tutoring  to  ET.  The  demonstration  is  not  implemented  on  the  system  but  on  a  simulation  of 
the  system.  The  simulation  runs  on  the  same  system  (a  multi-media  personal  computer) 
that  runs  EWIET  and  is  intended  to  emulate  shipboard  training. 
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ANNOTATED  POLICY  STATEMENTS  REGARDING 
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Department  of  Defense.  (1996,  7  October).  Defense  science  and  technology  plan. 
Washington,  DC:  Author. 

In  the  context  of  listing  deficiencies  and  barriers  to  joint  readiness,  the 
following  quote  is  relevant:  “The  principal  barriers  to  more  effective  joint 
and  combined  staff  training  include  the  lack  of  interoperability  among 
service  and  allied  training  simulations  and  models  and  the  lack  of  tools  and 
methods  for  assessment  and  feedback.  Another  barrier  is  the  absence  of  an 
embedded  training  capability  in  C4I  and  weapons  systems.” 


White,  John  P.  (1996,  23  September).  [Memorandum  from  the  Deputy  Secretary  of 
Defense.  Subject:  Accelerating  the  Application  of  Embedded  Training  and  Advanced 
Simulation  Technologies.]  Washington,  DC:  Author. 

This  memorandum  initiates  a  DoD-wide  effort  to  determine  how  we  can 
capture  the  potential  for  embedded  training  and  advanced  simulation 
technologies  to  reduce  overall  costs,  and  at  the  same  time,  improve 
readiness  and  increase  milit^  capability.  While  some  work  is  already 
underway,  we  must  place  a  higher  priority  on  finding  opportunities  to  apply 
these  and  other  promising  teclmologies  that  will  help  the  Department  reahze 
their  benefits  as  soon  as  possible ...  In  that  regard,  I  would  like  to  set  up  a 
Training  Coordination  Group  (TCG)  that  will  focus  on  identifying 
opportuiuties  to  apply  embedded  training,  advanced  simulation,  and  other 
technologies . . . 


WarNet  XXL  (1996,  13  August).  WarNet  XXI  action  plan  (Final  Coordinating  Draft). 
Washington,  DC:  Author. 

Army  policy  mandates  the  consideration  and  evaluation  of  embedded 
training  as  the  preferred  alternative  among  other  approaches  to  incorporating 
training  subsystems  in  the  development  of  all  Army  materiel  systems. 
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United  States  Anny  Training  and  Doctrine  Command.  (1996,  3  June).  Embedded  training 
concept  (Draft  TRADOC  Pamphlet  350-70-XX).  Washington,  DC:  Author. 

Summary:  This  concept  serves  as  the  basis  for  technological  and  doctrinal 
initiatives  that  support  the  Army’s  ability  to  train  using  capabilities  designed 
into  or  added  onto  its  operational  systems.  It  describes  embedded  training 
and  provides  the  framework  to  incorporate  it  into  Army  vision  and 
requirements  documents. 


Naval  Surface  Warfare  Center.  (1996,  May).  [Web  site  for  BFTT  Information  Net 
(BIN).]  Port  Hueneme,  CA:  Author, 

In  describing  the  basic  rationale  for  the  BFTT,  web  site  authors  contend  that 
it  is  an  outgrowth  of  the  Navy’s  Tactical  Training  Strategy,  which  itself  is 
based  on  team  training  research  and  empirical  studies  over  ^e  last  ten  years. 

This  strategy  was  articulated  in  the  CNO  message  of  December  1991: 

, .  the  ship  when  properly  supported  presents  the  most  effective  training 
site  for  appropriate  operational  and  functional  training.  This  allows  the 
ships  to  train  using  their  own  equipment,  system  configurations,  and 
operational/casualty  procedures.” 


CINCLANTFLT/CINCPACFLT.  (1996,  15  January).  Tactical  training  manual 
(Instruction  3501.1 A). 

In  developing  training  requirements  that  support  warfighting  capabilities, 
the  following  precepts  were  followed:  “Wherever  possible,  conduct 
training  on-board  ship  with  organic  training  devices  and  installed  equipment 
under  supervision  of  shipboard  training  teams”  (Chapter  5). 


Force  XXL  (1995, 23  October).  Warfighter  OPORD  1-95  (WARHGHTER  XXI).  Fort 
Knox,  KY:  Author. 

The  goal  of  WF  XXI  is  to  use  digital  technology  to  maintain  a  continuous 
edge  in  projecting  and  employing  combat  power  on  future  batdefields. 
Mirroring  this  effort  must  also  be  initiatives  to  embed  the  complex, 
combined  arms,  structured  training  of  the  future  into  the  digitized  force. 

The  end  state  is  the  “digitized  battlefield”  to  provide  seamless,  digital  C2 
capabilities  for  the  entire  fighting  force. 
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Force  XXI  (1995,  15  June).  Warfighter  XXI  (WFXXl).  Fort  Knox,  KY:  Author. 

Currently,  the  large  majority  of  all  TADSS  arc  non-system  devices.  The 
ultimate  goal  of  Warfighter  XXI  is  to  drive  the  development  of  the 
technology  that  will  support  fully  embedded  TADSS  in  the  prime  systems. 
This  shift  in  balance  between  system  and  non-system  TADSS  will  require  a 
closer  link  between  prime  system  materiel  development  and  TADSS 
development  (Chapter  5). 


Slatkin,  N.  (1995, 3  May).  [Statement  of  the  Assistant  Secretary  of  the  Navy  (Research, 
Development,  and  Acquisition)  before  the  House  Appropriations  Committee  on 
Navy/Marine  Corps  Acquisition.]  Washington,  DC;  Department  of  the  Navy. 

Hon.  Nora  Slatkin  highlighted  embedded  training  initiatives  in  her  proposed 
FY96  S&T  plan:  “The  computerized  nature  of  modem  naval  systems 
enables  us  to  embed  system-based  intelligent  training  technologies  into 
operational  systems.  These  training  systems  will  significantly  reduce  direct 
training  costs.  Additional  savings  will  be  realized  through  reduced  travel 
costs,  reduced  need  for  classrooms,  and  reduced  associated  logistics.  By 
utilizing  built-in  training  modes,  we  gain  greater  training  effectiveness  and 
the  flexibility  inherent  in  on-demand  training  capability  available  anywhere 
at  any  time.” 


Department  of  the  Army.  (1990,  30  May).  Army  modernization  training  (AR  350-35). 

Washington,  DC;  Author. 

The  training  developers  are  responsible  for  identifying,  planning,  and 
coordinating  the  conduct  of  training  to  support  Army  modernization.  The 
exact  form  of  this  training  will  be  determined  by  the  training  developer  in 
coordination  with  the  combat  developer.  The  use  of  embedded  training  to 
meet  these  needs  should  be  given  primary  consideration  .  .  .  training  that 
results  from  features  designed  and  buUt  into  a  specific  end  item  of 
equipment  to  provide  training  in  its  use.  It  will  not  interfere  with  the 
operational  requirements  capabilities  of  the  system,  and  it  trains  individual 
tasks  through  force  level  tasks  as  required. 


Joint  Technical  Coordinating  Group  for  Training  Systems  and  Devices  (JTCG-TSD). 
(1989).  [Study  plan  for  embedded  training.]  Washington,  DC:  Author. 

According  to  this  document  the  following  is  the  Air  Force’s  definition  of 
embedded  training;  “A  training  capability  which  is  designed  into  or  added 
onto  operational  equipment.” 
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C/NET  Task  Force  on  Embedded  Training.  (1985,  14  November).  [OPNAVINST  on 
embedded  training.]  Washington,  DC:  Department  of  the  Navy. 

Embedded  training  is  defined  as  “.  .  .  training  that  is  provided  by 
capabilities  built  into  or  added  onto  operational  systems,  subsystems,  or 
equipment,  to  enhance  and  maintain  the  skill  proficiency  of  fleet  personnel” 
(paragraph  4.1). 


Thurman,  M.,  and  Ambrose,  J.R.  (1987,  March).  [Policy  and  guidance  letter  from  Vice 
Chief  of  Staff  of  the  Army  and  the  Undersecretary  of  the  Army.  Subject:  embedded 
training.]  Washington,  DC:  Department  of  the  Army. 

Embedded  training  is  defined  as  “.  .  .  training  that  is  provided  by 
capabilities  designed  to  be  built  into  or  added  into  operation^  systems  to 
enhance  and  maintain  the  skill  proficiency  necessary  to  operate  and  maintain 
that  end  item  equipment.”  The  Army’s  unambiguous  policy  is  that  “.  .  .  ET 
will  be  included  in  all  new  and  developing  Army  systems  unless  there  are 
valid  and  compelling  reasons  not  to  do  so.” 
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APPENDIX  D. 

SUMMARY  DESCRIPTIONS  OF 
EMBEDDED  TRAINING  (ET)  SYSTEMS 


Service:  Air  Force 

ET  System  Name:  AN/GPS-T2 

Parent  System:  Plan  Position  Indicator  (PPI)  displays  of  Air  Defense  Direction 

Centers  (ADDCs) 

Functional  Description 

The  AN/GPS-TS  was  a  pioneering  ET  device  [circa  late  1950’s,  before  develop¬ 
ment  of  Semi-Automatic  Ground  Environment  (SAGE)]  that  strapped  onto  PPI  displays. 
The  AN/GPS-T2  converted  prerecorded  films  into  signals  that  were  interpreted  as  actual 
radar  target  blips.  This  is  an  example  of  ET  “stimulation.”  An  accessory  to  this  system, 
the  Anti-Countermeasures  Trainer  (ACTER)  OA-1767,  enabled  the  PPI  to  simulate  the 
effects  of  electronic  countermeasures  (ECMs). 

Assessment 

Results  from  an  evaluation  of  the  system  training  program  (STP),  of  which  the 
AN/GPS-T2  was  an  integral  part,  indicated  that  performance  of  ADDC  teams  who  trained 
on  STP  improved  more  than  teams  who  did  not  train  on  STP  (Jaffe,  1958).  Also,  the  deci¬ 
sion  to  fully  integrate  ET  into  the  successor  computer-based  system  (SAGE)  was  an  indi¬ 
cation  of  the  success  of  the  AN/GPS-T2. 

Positive  Features 

•  The  use  of  AN/GPS-T2  was  supported  by  an  all-encompassing  training  pro¬ 
gram  (STP)  that  included  other  features  designed  to  improved  training,  such  as 
a  standardized  debriefing  procedure  and  an  instructional  sequencing  strategy. 

Negative  Features 

•  Fidelity  was  limited  by  noninteractive  targets.  For  instance,  threat  bombers 
continued  to  move  on  their  predefined  path  even  after  they  had  been  “shot 
down.” 
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Service:  Air  Force 

ET  System  Name:  On  Board  Electronic  Warfare  Simulator  (OB EWS) 

Parent  System:  F-16  and  F-15E  aircraft 

Functional  Description 

OBEWS  is  a  strap-on  system  embodied  in  a  pod  strapped  under  the  wing  of  the  air¬ 
craft.  This  pod,  which  has  characteristics  of  a  advanced  medium  range  air-to-air  missile 
(AMRAAM),  produces  a  threat  simulation  to  radar  warning  receivers  in  flight.  OBEWS 
comprises  two  major  subsystems:  the  on-board  subsystem  (OBS)  and  ground  support 
subsystem  (GSS),  which  interact  via  an  electrically  erasable,  programmable  read  only 
memory  (EEPROM)  module.  Scenarios  are  loaded  into  the  EEPROM  module  at  the  GSS. 
After  a  training  flight,  data  downloaded  from  the  EEPROM  to  the  GSS  are  used  to  debrief. 

Assessment 

Initial  testing  (Glenn,  1989)  of  OBEWS  indicated  positive  responses  to  the  system. 
However,  despite  favorable  public  responses  to  the  system,  the  Tactical  Air  Command 
(TAC)  has  been  slow  in  adopting  the  system. 

Positive  Features 

•  Scenario  information  includes  terrain  data  that  simulate  the  effects  of  terrain 
masking. 

•  OBEWS  targets  can  be  mixed  with  simulated  targets  generated  from  ground 
sites. 

•  Pilots  can  react  as  they  would  to  electronic  warfare  (EW)  threats  (jamming, 
chaffi'flare  launch,  or  terrain  masking).  Responses  are  recorded  by  the 
EEPROM  module. 

•  GSS  provides  detailed  and  flexible  replay.  It  allows  compressed  or  real-time 
replay,  comparison  of  the  actual  and  the  planned  route,  threat  events,  radar 
warning  receiver  (RWR)  audio  and  symbology — ^all  of  which  are  correlated 
with  aircraft  position,  attitude,  terrain,  and  countermeasures  status. 

•  Replay  can  be  provided  in  standard  planar  view,  two  dimensional  (2-D) 
shaded  relief,  or  a  three  dimensional  (3-D)  view. 

•  Conversion  time  from  training  to  combat  operations  mode  is  minimal, 
approximately  1  hour. 

•  In  contrast  to  range-based  simulations,  OBEWS  is  nonradiating;  thus,  the 
system  is  invisible  to  electronic  surveillance  from  potential  adversaries. 


D-4 


Negative  Features 

•  For  safety  considerations,  only  one  aircraft  can  be  trained  at  a  time.  That  is, 
OBEWS  does  not  simulate  a  situation  where  multiple  aircraft,  flying  in  close 
formation,  are  illuminated  by  a  single  threat  radar  beam. 

•  The  simulation  does  not  accommodate  electronic  counter-countermeasure 
(ECCM)  capabilities. 

•  The  pod  degrades  the  aerodynamic  characteristics  of  the  aircraft. 

•  While  fitted  with  OBEWS,  the  aircraft  is  not  combat  operational. 


Service: 


Air  Force 


ET  System  Name:  Semi- Automated  Ground  Environment  (SAGE) 

Parent  System:  Semi-Automated  Ground  Environment  (SAGE) 

Functional  Description 

The  computer-based  SAGE  Direction  Centers  carried  out  the  functions  of  several 
manual  ADDCs.  SAGE  computers  had  embedded  in  them  the  ability  to  run  computerized 
simulation  scenarios  through  the  system  displays. 

Assessment 

Rowell  (1962,  cited  in  Parsons,  1972)  tracked  the  performance  of  four  crews,  each 
comprising  approximately  50  officers  and  airmen,  as  they  trained  on  SAGE  over  a 
2.5-month  period.  Even  though  scenarios  were  systematically  increased  in  difficulty  over 
that  time,  a  variety  of  performance  measures  indicated  substantial  improvements  over 
repeated  practice  trials.  Unfortunately,  this  evaluation  did  not  sort  out  the  effects  of  the  ET 
aspects  of  SAGE  from  the  other  aspects  of  the  Systems  Training  Program  (STP)  in  which 
the  ET  system  was  used. 

Positive  Features 

•  Like  the  manual  STP  program,  the  SAGE  ET  system  was  part  of  a  compre¬ 
hensive  instructional  system. 

•  The  SAGE  system  captured  and  processed  some — ^but  not  all — performance 
data  needed  for  debrie^g. 

Negative  Features 

•  SAGE  ET  system  required  substantial  human  trainer  support. 


D-6 


Service: 


Air  Force 


ET  System  Name: 

Parent  System:  World-Wide  Military  Command  and  Control  System  (WWMCCS) 

Functional  Description 

WWMCCS  is  an  older  Joint-Service  command  and  control  (C2)  system  that  was 
developed  by  the  Air  Force.  The  ET  is  fully  embedded  in  the  system’s  general-purpose 
computers,  but  it  is  used  in  training  (as  opposed  to  operational)  mode  only.  The  ET  system 
incorporates  simulation  scenarios  and  computer-based  tutorials. 

Assessment 

Warm  et  al.  (1988)  did  not  provide  as  much  information  about  the  WWMCCS  as 
they  did  with  other  systems  in  their  review.  Nevertheless,  they  characterize  the  system  as 
highly  reliable  and  available  for  training  within  the  constraints  of  operational  needs.  Users 
rate  it  as  easy  to  use,  with  fast  start-up  and  shut-down  times.  The  individual  computer- 
based  modules  are  used  often,  much  more  so  than  the  computer-based  scenarios. 

Positive  Features 

•  The  tutorials  can  run  at  some  stations  while  the  system  continues  to  operate  on 
others. 

•  The  ET  system  includes  a  library  of  canned  scenarios  and  computer-based 
instruction  (CBI)  modules. 

•  Switching  between  training  and  operation  mode  is  fast  (less  than  1  minute). 

•  Units  can  author  and  store  computer-based  tutorials. 

•  Scenarios  can  be  run  over  the  WWMCCS  network,  thereby  providing  training 
in  coordination  among  locations. 

•  The  system  has  record-keeping  capabilities  that  are  used  to  determine  the 
readiness  of  the  WWMCCS  network. 

•  Computer-based  tutorials  have  automated  performance  measurement 
capabilities. 

Negative  Features 

•  Units  caimot  author  simulation  scenarios. 

•  Scenarios  do  not  have  automated  performance  measurement  capabilities. 

•  Computer-based  scenarios  have  no  freeze,  playback,  or  fast-forward 
capabilities. 
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Service: 


Air  Force 


ET  System  Name:  Onboard  Simulation  (OB  S) 

Parent  System:  F-15B  Flight  Control  System 

Functional  Description 

Sponsored  by  the  Air  Force  Wright  Aeronautical  Laboratories,  OBS  was  originally 
designed  for  flight  testing  of  the  Integrated  Flight  and  Fire  Control  (IFFC)  system  on  the 
F-15B,  which  occurred  between  July  1981  and  January  1983.  It  was  originally  conceived 
as  an  alternative  to  a  “hot  bench”  unit,  which  is  a  test  stand  parked  next  to  aircraft  and  used 
to  generate  test  signals  to  aircraft  subsystems. 

OBS  software  is  fully  embedded  in  one  of  the  aircraft’s  digital  computers  and  gen¬ 
erates  dynamic  targets  via  the  pilot’s  heads-up  display  (HUD).  It  simulates  air-to-air  and 
air-to-ground  gunnery  and  bombing  operations.  It  normally  used  while  in  transit  in  a  non- 
tactical  environment  but  can  be  used  on  the  ground. 

Assessment 

As  of  the  writing  of  Warm  et  al.  (1988),  the  OBS  was  an  experimental  system  and 
had  not  been  implemented  in  all  active  Air  Force  squadrons.  In  units  where  it  was  imple¬ 
mented,  however,  users  rate  the  OBS  as  easy  to  use,  with  a  single  switch  for  engaging  or 
disengaging  the  system  in  less  than  1  minute. 

Kocher  (1984)  documented  two  types  of  savings  that  OBS  accrued  during  IFFC 
testing.  First,  it  reduces  or  eliminates  the  costs  associated  with  live  training,  including 
those  related  to  towed  or  tactical  targets  and  to  ammunition  and  bombs.  Second,  it 
increases  the  number  of  encounters  (“events”)  that  can  be  performed  per  flight  hour.  With 
OBS,  the  system  is  simply  reset  after  each  encounter  to  prepare  for  the  next.  With  real  tar¬ 
gets,  lengthy  set-up  time  is  required  to  position  the  target  and  the  attacker  after  each  run. 

Positive  Features 

•  Hie  OBS  provides  onboard  scoring  using  the  Air  Combat  Evaluator  (ACE) 
software.  Information  is  provided  to  the  pilot  via  a  Bullets-At-Target-Range 
(BATR)  symbol  (or  “hot  point”).  For  air-to-ground  scenarios,  B  ATR  displays 
the  pilot’s  bullet  stream  relative  to  the  target.  Also,  during  air-to-ground  sce¬ 
narios,  BATR  and  ACE  are  used  to  compute  predicted  bullet  hits  and  miss 
distance  for  bombing  runs. 

•  OBS  allows  the  simulation  of  head-on  air-to-air  encounters  without  the  risk  of 
mid-air  collision. 

•  Activation  and  deactivation  of  OBS  software  takes  less  than  1  minute. 

•  OBS  includes  an  extensive  library  of  randomly  accessed  canned  scenarios. 

•  OBS  targets  are  more  evasive  than  towed  live  targets  can  be. 
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Negative  Features 

•  Visuals  are  to  the  field  of  view  from  the  HUD.  This  seriously  limits  training 
for  air-to-air  combat. 

•  OBS  does  not  simulate  nonvisual  stimuli  that  accompany  firing  (vibration  and 
sound). 

•  The  system  has  no  scenario  authoring  feature.  Scenarios  can  only  be  changed 
with  avionics  system  upgrades. 

•  The  system  has  no  on-line  scenario  modification  feature. 

•  Performance  feedback  is  not  stored  and  cannot  be  printed  out 

•  The  system  does  not  have  scenario  freeze,  replay,  or  fast-forward  features. 

•  When  OBS  is  used,  other  aircraft  are  not  allowed  in  airspace  for  safety  rea¬ 
sons.  Thus,  multi-aircraft  tasks  cannot  be  trained. 

•  OBS  requires  substantial  training  to  operate  system. 

•  OBS  cannot  be  used  when  aircraft  is  in  operational  status.  Aircraft  must  be 
dedicated  to  training. 

•  OBS  does  not  train  missile  firing  tasks. 
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Service: 


Air  Force 


ET  System  Name:  Airborne  Operational  Computer  Program  (AOCP) 

Parent  System:  Airborne  Warning  and  Control  System  (AW ACS) 

Functional  Description 

AOCP  software,  which  is  fully  integrated  into  AWACS  computers,  presents  simu¬ 
lated  scenarios  on-  or  off-line.  The  AOCP  presents  simulation  scenarios  through  the  six 
situation  display  consoles  (SDCs)  on  the  aircraft.  ET  is  normally  used  after  an  aborted 
mission — ^when  a  subsystem  (e.g.,  radar)  fails  and  prevents  the  completion  of  the  mission. 

Assessment 

AWACS  is  a  reliable  system  and  is  normally  available  for  operations  and  training, 
although  training  access  can  be  restricted  by  flight  demands.  “AWACS  ET  is  easy  to  oper¬ 
ate  from  startup  to  shutdown”  (Warm  et  al.,  1988,  p.  F-4). 

Positive  Features 

•  In  addition  to  off-line  training,  training  can  occur  while  aircraft  is  in  the  air  and 
the  AWACS  system  is  in  on-line,  with  no  serious  decrement  to  readiness.  If 
on-line,  the  AOCP  is  automatically  deactivated  when  AWACS  receives  an 
alert 

•  AOCP  can  use  canned  scenarios. 

•  Crews  can  author  and  store  their  own  scenarios. 

•  Scenario  replay  includes  operator  responses. 

•  System  has  scenario  freeze,  replay,  and  fast-forward  capabilities. 

•  The  system  takes  less  than  1  minute  to  return  from  the  training  to  the  opera¬ 
tional  mode. 

•  AWACS  can  send  simulated  target  data  to  other  aircraft,  ground  stations,  or 
shipboard  C2  systems,  thereby  enabling  training  in  coordination. 

Negative  Features 

•  Uploading  scenarios  takes  a  moderate  amount  of  time  (at  least  51  minutes). 

•  Scenarios  cannot  be  modified  once  started. 

•  Other  than  replay,  system  has  no  automated  performance  measurement. 
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Service:  Air  Force 

ET  System  Name:  Sentinel  Byte  Prototype  Tutor 
Parent  System:  Sentinel  Byte 

Functional  Description 

Sentinel  Byte  is  an  Air  Force  command,  control,  communications,  and  intelligence 
(C3I)  system  for  planning  air  routes  and  analyzing  air  defense  (AD)  threats.  The  prototype 
tutor,  developed  by  Mitre  Corporation,  provides  an  overview  to  the  system,  instructs 
operators  on  specific  procedures  for  using  Sentinel  Byte  to  generate  the  threat  picture,  and 
provides  on-screen  checklists. 
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Service:  Air  Force 

ET  System  Name:  Computerized  On-line  Advising  and  Contextual  Help  (COACH) 

Parent  System:  Portable  Maintenance  Aid  (PMA)  of  the  Integrated  Maintenance 

Information  System  (IMIS) 

Functional  Description 

COACH  is  a  software  application  that  can  run  on  a  personal  computer  (PC)  as  well 
as  on  its  primary  medium,  the  PMA.  The  PMA  is  a  lightweight,  hardened  portable  com¬ 
puter  for  use  on  the  flight  line.  Although  the  PMA  is  limited  by  its  small  screen,  poor  pres¬ 
entation  quality,  and  limited  keyboard,  it  has  access  to  the  most  of  the  components  of 
IMIS. 

COACH  is  designed  to  be  used  in  the  flight-line  maintenance  environment  but  not 
while  maintenance  is  actually  being  conducted — ^that  is,  it  is  designed  as  a  trainer,  not  as  a 
job  aid,  even  thought  it  is  embedded  in  one.  COACH  is  structured  to  present  three  levels 
of  training:  apprentice,  journeyman,  and  craftsman.  These  different  training  levels  vary  in 
the  amount  of  interaction  between  student  and  his  virtual  mentor,  the  “Coach.”  Although  it 
is  intended  to  be  used  in  the  PMA  in  a  flight-line  maintenance  environment,  its  portability 
across  computer  media  enlarges  its  application  to  other  environments,  such  as  classroom, 
ready  room,  and  even  home  study. 

As  described  by  Wilson  et  al.  (1996),  it  is  a  demonstration  project  to  illustrate  the 
training  potential  of  IMIS  using  the  PMA.  COACH  currently  exists  as  a  detailed  descrip¬ 
tion  of  requirements  and  suggested  screens  that  could  be  used. 
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Service:  Army 

ET  System  Name:  Abrams  Full  Crew  Interactive  Simulator  Trainer  (AFTST) 

Parent  System:  Ml -series  tanks 

Functional  Description 

AFIST  is  an  ET  system  that  is  appended  to  a  powered-down  Ml  tank.  AFIST  uses 
commercial  off-the-shelf  (COTS)  technology  to  present  computer-generated  graphics  and 
sounds  to  the  turret  and  driver’s  compartment.  All  four  tank  crewmen  use  tank  controls  to 
practice  precision  and  degraded  gunnery  skills. 

Positive  Features 

•  The  system  contains  500+  tactical  engagements  derived  from  the  Tank  Gun¬ 
nery  tables  (FM  17-12-1). 

•  The  system  provides  a  detailed  critique,  which  can  be  printed  out  for  the  crew. 

•  The  AFIST  training  management  system  includes  a  training  matrix  that  organ¬ 
izes  the  engagements  into  a  progressively  sequenced  exercises  and  maintains  a 
crew  training  history. 

Negative  Features 

•  Instructors  cannot  author  new  engagements. 
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Service:  Army 

ET  System  Name:  Integral  Operator  Trainer  (lOT) 

Parent  System:  Homing  All-the-Way  Killer  (HAWK)  Air  Defense  Weapon  System 

Functional  Description 

The  lOT  is  a  simulator  embedded  in  the  HAWK  system.  Raytheon  has  developed  a 
laptop  computer-based  system  called  the  Advanced  Scenario  Generator  (ASG).  The  ASG 
interfaces  with  the  lOT. 
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Service:  Aniiy 

ET  System  Name:  Intelligent  Embedded  Operator  Assistant  (lEO A) 

Parent  System:  HAWK  Air  Defense  Weapon  System 

Functional  Description 

lEOA  is  an  intelligent  tutor  that  accompanies  the  HAWK  embedded  simulator — ^the 
lOT.  Operational  since  1987,  the  lEOA  is  regarded  as  one  of  the  first  real-world  applica¬ 
tions  of  intelligent  tutoring  systems. 
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Service; 


Army 


ET  System  Name:  Live  Air  Trainer  (LAT) 

Parent  System:  PATRIOT  Missile  System 

Functional  Description 

LAT  enables  PATRIOT  crews  to  use  disabled  systems  to  engage  real  targets.  It 
simulates  missile  firing,  missile  flyout,  and  engagement  of  targets  of  opportunity  and  their 
destruction.  LAT  was  scheduled  to  be  available  in  the  spring  of  1986  (Strasel  et  al.,  1988). 
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Service:  Army 

ET  System  Name: 

Parent  System:  Forward  Area  Air  Defense,  Command,  Control,  and  Intelligence 

(FAADC2I) 

Functional  Description 

FAAD  C2I  is  an  integrated  computer  hardware,  software,  and  commimications 
system  for  identifying  aircraft,  distributing  C2I  data  among  Air  Defense  Artillery  (ADA) 
units  and  combined  arms  elements,  giving  early  warning  to  rest  of  force,  and  alerting  Air 
Force  and  rear  area  defenses  about  transiting  enemy  aircraft.  The  system  provides  an 
embedded  simulation  capability  that  replicates  situations  encountered  in  mission  operation. 
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Service:  Anny 

ET  System  Name:  Air  Ground  Engagement  System  (AGES) 

Parent  System:  AH-64  (Apache),  OH-58  (Kiowa),  and  UH-60  (Blackhawk)  heli¬ 

copters 

Functional  Description 

AGES  is  an  appended,  eye-safe  laser  simulation  for  simulating  the  effects  of  Hell- 
fire  air-to-ground  missiles.  It  also  allows  aircraft  to  be  engaged  by  ground  weapons  and 
other  aircraft.  As  a  member  of  the  MILES  family  of  devices,  it  is  compatible  with  simula¬ 
tion  systems  (e.g.,  MILES  2000)  used  at  Combat  Training  Centers  (CTCs). 

Positive  Features 

•  AGES  n  records  events  for  After- Action  Review  (AAR)  purposes,  identifying 
by  player  and  weapon  type. 

•  System  simulates  aircraft  survivability  equipment. 
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Service:  Army 

ET  System  Name:  Training  Interface  Unit  (TIU) 

Parent  System:  Aquila  Remotely  Piloted  Vehicle  (RPV) 

Functional  Description 

Designed  by  Rediffusion  and  Silicon  Graphics,  the  TIU  straps  onto  the  system  to 
provide  the  additional  computing  capacity  needed  to  generate  simulated  imagery  through  the 
Aquila  system  display.  The  Aquila  ET  system  trains  the  RPV  mission  commander,  air 
vehicle  operator,  and  mission  payload  operator. 
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Service:  Amy 

ET  System  Name:  Troop  Proficiency  Trainer  (TPT) 

Parent  System:  PATRIOT  Missile  System 

Functional  Description 

Fully  integrated  into  the  PATRIOT’S  computer  systems,  the  TPT  simulates  actual 
PATRIOT  data  while  disabling  the  radar  system — that  is,  it  is  strictly  an  off-line  device. 
Each  battery  can  train  individually  or  be  networked  to  simulate  battalion-directed  AD  func¬ 
tions.  The  TPT  mimics  the  institutional  trainer  (P-COFT),  from  which  it  can  port  simula¬ 
tion  scenarios. 

Assessment 

The  TPT  is  generally  perceived  as  a  success — that  is,  a  well-designed  ET  system 
that  is  used  often  in  garrison  settings  (Strasel  et  al.,  1988).  The  users  in  Warn  et  al.’s 
(1988)  study  indicated  that  the  TPT  was  easy  to  use  and  had  relatively  short  start-up  and 
shut-down  times. 

Positive  Features 

•  ET  was  designed  into  the  system. 

•  ET  has  minimal  impact  on  system  random  access  memory  (RAM). 

•  Transition  from  training  to  operational  mode  and  from  operational  mode  to 
training  mode  is  rapid  (matter  of  minutes). 

•  The  TPT  includes  a  library  of  “canned”  scenarios  plus  the  ability  to  author  new 
ones. 

•  The  system  provides  feedback  in  the  form  of  printouts  that  summarize 
engagement  outcomes. 

Negative  Features 

•  Activation  of  ET  disables  radar  and  launchers. 

•  Feedback  does  not  include  information  on  operator  responses. 

•  Tbe  library  contains  a  relatively  small  number  of  canned  scenarios,  which 
means  that  users  can  quickly  “learn  the  scenarios.” 
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Service:  Army 

ET  System  Name: 

Parent  System:  All  Source  Analysis  System  (ASAS) 

Functional  Description 

ASAS  is  the  Army  version  of  a  joint  system  for  processing  intelligence  information 
and  corresponds  to  the  Air  Force’s  Enemy  Situation  Correlation  Element  (ENSCE).  Plans 
were  made,  at  least  in  1988,  to  embed  a  training  system  into  ASAS. 

Assessment 

Evans  et  al.  (1988)  derived  lessons  learned  from  the  design  of  ET  for  ASAS/ 
ENSCE.  However,  they  provided  no  assessment  of  the  resulting  system. 
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Service: 


Army 


ET  System  Name: 

Parent  System:  Advanced  Field  Artillery  Tactical  Data  System  (AFATDS) 

Functional  Description 

AFATDS  was  designed  as  a  replacement  for  the  Tactical  Fire  Direction  System 
(TACFIRE).  Its  computer  systems  are  more  compact  and  user  friendly.  It  provides  the 
fire  support  portion  of  the  Aimy  Tactical  Command  and  Control  System  (ATCSS),  using 
its  common  hardware  components  and  common  operating  environment  (COE).  AFATDS 
training  includes  both  stand-alone  and  embedded  systems.  The  embedded  system  has  not 
been  described  in  any  detail. 
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Service:  Army 

ET  System  Name:  Aircraft  Survivability  Equipment  Trainer  (ASET)  HI 
Parent  System:  Aircraft  Survivability  Equipment  (ASE) 

Functional  Description 

The  ASET  m  is  a  subsystem  installed  in  Army  aircraft  or  flight  simulators  to  pro¬ 
vide  replication  of  AD  threats  during  routine  missions.  In  a  training  role,  the  ASET  El 
activates  the  ASE,  allowing  aviators  an  opportunity  to  operate  various  countermeasures  and 
employ  appropriate  tactics,  techniques,  and  procedures  (TT&P).  Real-time  feedback  is 
provided  to  the  aviator  for  learning  and  AARs. 
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Service:  Army 

ET  System  Name: 

Parent  System:  Combat  Service  Support  Command  System  (CSSCS) 

Functional  Description 

CSSCS  is  the  commander’s  combat  service  support  (CSS)  C2  system.  It  interfaces 
with  the  family  of  Standard  Army  Management  Information  Systems  (STAMIS)  currently 
fielded  to  the  Army.  A  compact  disk  (CD)-based  ET  system  provides  three  modes  of 
training-  training  course  (tutorial),  scenario  training  (simulated  batdefield  environment), 
and  free  play. 
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Service; 


Army 


ET  System  Name: 

Parent  System:  Tactical  Fire  Direction  System  (TACFIRE) 

Functional  Description 

The  TACFIRE  ET  system  has  two  functions.  First,  it  provides  computer-based 
tutorials — ^which  were  developed  in  PLANTT  authoring  language — ^for  training  TACFIRE 
operators.  Second,  it  presents  simulation  scenarios  for  training  small  teams  of  computer 
and  acronym  operators  in  TACFIRE  messages.  The  scenarios  implemented  on  tape  require 
these  teams  to  complete  and  transmit  TACFIRE  messages  correctly. 

Assessment 

The  TACFIRE  ET  system  received  the  most  negative  evaluation  of  all  systems 
evaluated  by  Strasel  et  al.  (1988).  The  TRADOC  Systems  Analysis  Activity  (TRASANA) 
documented  that  only  a  small  minority  of  TACFIRE  operators  use  it  or  have  used  it. 
Germas  and  Baker  (1980)  showed  no  difference  between  the  computer-assisted  instruction 
(CAI)  mode  of  TACFIRE  and  the  traditional  mode  of  instruction  on  unmediate  and  delayed 
paper-and-pencil  knowledge  test,  and  they  indicated  positive  attitudes  toward  the  system 
during  its  initial  assessment. 

Positive  Features 

•  Although  designed  after  the  fact,  the  ET  system  is  completely  embedded  in 
TACHRE. 

•  The  TACFIRE  is  a  large  computer  system  that  is  well  suited  to  ET. 

Negative  Features 

•  TACFIRE  ET  uses  the  same  user  interface  for  training  that  is  used  for  opera¬ 
tions.  This  interface  has  been  often  criticized  as  not  being  user  friendly. 
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Service:  Amy 

ET  System  Name:  Howitzer  Improvement  Program  Embedded  Training 
(HIP  ET) 

Parent  System:  Howitzer  Improvement  Program  (HIP):  M 1 09 A6  self-propelled 

howitzer 

Functional  Description 

HIP  ET  is  fully  embedded  in  and  delivered  through  the  Automated  Fire  Control 
System  (AFCS),  which  includes  a  visual  display  and  26  keys  and  switches  to  control  the 
system  and  input  data.  The  HIP  ET  presents  computer-based  tutorials  and  simulated  sce¬ 
narios.  Both  are  intended  for  training  individual  artillerymen. 

Assessment 

Schopper  et  al.’s  (1990)  abbreviated  assessment  of  HIP  ET  indicated  positive  sub¬ 
jective  reactions  to  system  during  initial  testing.  Results  from  perfomance  tests  show 
soldiers  with  less  ability  committed  more  errors  and  took  more  time  to  complete  computer- 
based  tutorials.  Feedback  indicated  that  additional  information  and  practice  are  needed  to 
transition  from  tutorials  to  scenarios. 
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Service:  Army 

ET  System  Name:  Multiple  Integrated  Laser  Engagement  Simulation 

(MILES) 

Parent  System:  Armored  fighting  vehicles 

Functional  Description 

The  MILES  is  a  family  of  training  systems  that  simulate  the  effects  of  direct  fire 
weapons.  MILES  provides  the  capability  for  force-on-force  (EOF)  and  target  engagement 
in  a  real-time  environment  The  MILES  component  for  armored  fighting  vehicles  (Ml-, 
M2,  and  M3-series)  is  a  strap-on  device  for  emitting  and  detecting  laser  signals.  When  a 
vehicle  fires  its  direct  fire  system  at  an  opposing  vehicle,  the  lasers  replicate  the  trajectory 
of  live  ammunition.  The  receiver  on  the  target  vehicle  picks  up  the  firing  laser  and  affects 
the  laser  system  of  the  targeted  vehicle — depending  on  the  accuracy  of  the  fires,  the  type  of 
ammunition,  weapons  systems  involved,  a  probability  of  kill  factor,  and  several  other 
factors  like  range  and  angle  of  fire.  A  new  version  of  MILES  (MILES  2000)  takes  advan¬ 
tage  of  new  technology  to  produce  a  smaller,  more  invisible  system  that  includes  perform¬ 
ance  measurement  capabilities. 

Assessment 

Compared  to  earlier  methods  of  Tactical  Engagement  Simulation  (TES),  MILES 
drastically  reduces  the  number  of  observer/controllers  (0/Cs)  required  to  observe  and  score 
performance  (“training  overhead”).  ARI  was  involved  in  the  initial  user  acceptance  testing. 
They  also  participated  .  in  a  later  follow-up,  after  fielding,  which  identified  specific 
problems  the  units  were  having,  and  worked  with  the  TSM-TES  to  correct  them.  This 
implementation  plan  and  effort  for  MILES-TES  was  the  most  complete  and  effective  ever 
conducted  by  TRADOC”  (ARI  Newsletter,  Spring  1995).  The  Project  Director  for  MILES 
2000  has  stated  on  his  Web  page  that  “. . .  MILES  training  has  been  proven  to  dramatically 
increase  the  combat  readiness  and  fighting  effectiveness  of  military  forces”  (3  September 
1996). 

Positive  Features 

•  MILES  2000  will  have  a  performance  measurement  capability. 
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Service: 


Army 


ET  System  Name: 

Parent  System:  Army  Battle  Command  System  (ABCS) 

Functional  Description 

ET  is  implemented  in  one  of  three  ABCS  components:  the  Army  Tactical  Com¬ 
mand  and  Control  System  (ATCCS),  which  is  comprised  of  five  Battlefield  Functional 
Area  Command  and  Control  Systems  (BFACS).  Each  BFACS  has  on-line  help  features, 
mini  tutorials,  and  embedded  interactive  simulation  scenarios,  as  well  as  technical  manuals 
(TMs)  and  field  manuals  (FMs)  on  electronic  media. 
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Service;  Army 

ET  System  Name:  Improved  Target  Acquisition  System-Training  System  (I-TS) 
Parent  System:  Improved  Target  Acquisition  System  (ITAS) 

Functional  Description 

ITAS  is  an  add-on  product  improvement  to  the  Tube-launched,  Optically-tracked, 
Wire-guided  missile  (TOW)  2A/2B.  lATS  operates  in  two  modes:  (1)  the  CT  mode, 
where  the  system  can  be  used  by  itself  to  provide  sustainment  gunnery  training  and  (2)  the 
tactical  engagement  simulation  mode,  which  provides  capability  to  conduct  FOF  engage¬ 
ments  with  other  MILES  11/2000  systems.  lATS  presents  simulated  visual  effects  (i.e., 
launch  obscuration,  flyout,  and  burst  on  target)  in  gunner’s  sight. 
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Service:  Anny 

ET  System  Name: 

Parent  System:  Advanced  Tank  Armament  System  (ATAS)  for  Abrams  tanks 

Functional  Description 

ATAS  is  a  collection  of  technologies  for  improving  the  weapon  system  capability  of 
Abrams  tanks.  ATAS  includes  development  of  a  new  gun,  thermal  imaging  system, 
extended  range  fine  control  system  (FCS),  and  ET.  Texas  Instruments  (TI)  is  developing 
the  ET  component,  using  digitized  video  displayed  to  each  crew  station.  ET  concepts  are 
now  under  test  in  an  ARSI  testbed  for  evaluating  future  armored  vehicle  concepts. 
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Service: 


Army 

ET  System  Name:  Precision  Gunnery  System  (PGS) 

Parent  System:  Bradley  Fighting  Vehicle 

Functional  Description 

The  purpose  of  this  strap-on  ET  system  is  to  introduce  precision  gunnery  effects 
and  procedures  in  FOF  tactical  training.  The  system  simulates  25-mm  main  gun,  coaxial 
machine  gun,  and  TOW  missile  effects.  Fielding  of  PGS  began  in  May  1995. 
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Service:  Army 

ET  System  Name:  Tank  Weapons  Gunnery  Simulation  System  (TWGSS) 

Parent  System:  Abrams  Tank 

Functional  Description 

The  purpose  of  this  strap-on  ET  system  is  to  introduce  precision  gunnery  effects 
and  procedures  in  EOF  tactical  training.  The  system  simulates  main  gun  and  coaxial 
machine  gun  effects.  Fielding  of  TWGSS  began  in  May  1995. 


D-32 


Service:  Army 

ET  System  Name:  TPQ-29  Trainer 

Parent  System:  Improved  HAWK  (IHAWK)  Missile  System 

Functional  Description 

The  TPQ-29  is  a  strap-on,  umbilical  ET  system  that  was  developed  long  after  the 
operational  system  had  been  implemented.  The  ET  system — equipped  with  its  own  con¬ 
sole  for  initializing  and  controlling  training  sessions — is  mounted  in  a  van  that  is  connected 
by  cables  to  the  operational  equipment  One  system  is  issued  per  battalion  and  periodically 
rotated  among  batteries.  Batteries  are  typically  scheduled  for  two  16-hour  training  periods 
per  month. 

Assessment 

Warm  et  al.  (1988)  documented  several  problems  that  apply  to  the  TPQ-29. 
Because  of  strict  State  of  Readiness  (SOR)  requirements,  the  IHAWK  system  cannot  often 
be  taken  off-line  for  training.  In  some  cases,  actual  hook-up  times  have  been  longer  than  a 
battery  has  for  training.  The  IHAWK  system  must  be  retuned  after  use  of  TPQ-29.  Low 
reliability  of  IHAWK  has  also  limited  the  amount  of  time  available  for  training.  The  strap- 
on  nature  of  system  has  increased  logistic  requirements,  adding  unique  parts  and  systems 
to  IHAWK. 

Positive  Features 

•  TPQ-29  has  scenario  authoring  capability. 

Negative  Features 

•  TP(3-29  can  only  be  operated  when  the  IHAWK  system  is  off-line. 

•  System  has  no  on-line  control  of  scenario. 

•  It  has  no  performance  measurement  capabilities. 

•  It  does  not  communicate  with  other  units  to  simulate  coordinated  tasks. 

•  Signals  for  TPQ-29  are  lower  in  intensity  that  operational  signals,  requiring  the 
user  to  increase  the  signal  gain.  Users  think  that  this  detunes  the  system, 
which  contributes  to  their  dislike  of  the  system. 

•  System  has  no  scenario  freeze,  fast-forward,  or  replay  capabilities. 

•  Hook-up  pins  are  difficult  to  use  and  susceptible  to  bending  and  breaking. 
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Service;  Army 

ET  System  Name:  RAID  tapes 

Parent  System:  Missile  Minder  Command  and  Control  (C2)  System 

Functional  Description 

The  Missile  Minder  is  a  C2  system  that  controls  NIKE,  HAWK,  and  PATRIOT 
systems  at  the  AD  battalion  or  group  level.  The  ET  system  is  embodied  in  so-called  RAID 
tapes  that  store,  in  tape  cartridge  form,  scenarios  for  training.  A  second  magnetic  tape  unit 
records  results  from  simulated  incidents.  The  ET  system  is  used  for  training  Missile 
Minder  teams  and  performing  readiness  evaluations. 

Assessment 

According  to  Warm  et  al.  (1988),  users  judge  the  Missile  Minder  ET  to  be  easy  to 
use  and  quick  to  start-up  and  shut-down.  However,  the  system  is  only  infrequently  avail¬ 
able  for  training  (less  than  three  times  per  month). 

Positive  Features 

•  Simulation  fidelity  is  considered  very  good,  with  no  notable  difference 
between  simulated  and  operational  imagery. 

•  Performance  data  can  be  stored  and/or  printed  out. 

•  A  library  of  canned  scenarios  is  provided  with  the  system. 

•  System  has  playback  feature,  by  restarting  tape  (but  only  at  the  beginning). 

Negative  Features 

•  Although  ET  system  can  be  used  on-line  in  concept,  it  is  not  used  on-line  in 
practice  because  it  would  input  simulated  targets  as  real  targets. 

•  Users  do  not  have  ability  to  author  scenarios. 

•  Performance  data  is  limited  to  system  hits  and  misses. 

•  System  has  no  scenario  freeze  or  fast-forward  capabilities. 
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Service: 


Army 


ET  System  Name: 

Parent  System:  Crusader  self-propelled  howitzer  and  resupply  vehicles 

Functional  Description 

The  Crusader  is  a  155  mm  howitzer  system  currently  under  development.  It  com¬ 
prises  a  self-propelled  gun  and  independently  controlled  resupply  vehicle.  Designed  to 
replace  the  Palladin,  the  Crusader  will  incorporate  an  advanced  gun  propulsion  system 
Giquid  propellant),  automated  ammunition  handling,  rearm  and  refuel  under-armor  capabil¬ 
ity,  advanced  fire  control,  and  diagnostic/prognostics.  In  particular,  the  Crusader  system 
will  have  ET  for  individual  and  crew  tasks  to  reduce  the  need  for  live-fire  training.  The  ET 
system  will  be  Distributed  Interactive  Simulation  G!)IS)-compatible  so  that  the  vehicles  can 
participate  in  EOF  simulations  and  Synthetic  Theater  of  War  (STOW)  environments. 

In  addition,  the  system  will  include  an  embedded  decision  aid  to  automate  routine 
information  processing  functions,  while  helping  the  crew  anticipate  problems — such  as  low 
ammunition  or  fuel  levels — and  providing  appropriate  courses  of  action.  The  aid  will  con¬ 
stantly  evaluate  the  factors  of  mission,  enemy,  troops,  terrain,  and  time  (METT-T)  but  will 
filter  the  information  to  the  crew,  providing  only  what  they  need  to  know.  The  system  is 
already  being  tested  in  soldier-in-the-loop  simulations.  The  date  of  the  first  unit  to  be 
equipped  with  Crusaders  is  FY  2(X)5. 
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Service: 


Joint 


ET  System  Name: 

Parent  System:  Joint  Communications  Planning  and  Management  System  (JCPMS) 

Functional  Description 

The  JCPMS  will  provide  an  automated,  systems  management  capability  to  assist  the 
Conunanders  in  Chief  (CINCs),  Combined  Joint  Task  Forces  (CJTFs),  and  Joint  Task 
Force  (JTF)  component  Commanders  in  communications  planning  and  execution.  The 
mission-needs  statement  for  this  system  states  that  sustainment  training  needs  will  be  met 
with  ET  (on-line  help  features)  or  “the  most  cost  effective  media.” 
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Service: 


Joint 


ET  System  Name: 

Parent  System:  Joint  CoUection  Management  Tools  (JCMT) 

Functional  Description 

JCMT  is  the  DoD  Intelligence  Information  System  (DoDIIS)  migration  system  for 
all-source  collection  management  National,  theater,  and  tactical  organizations  of  all  Serv¬ 
ices  will  use  it  It  is  a  software-only  system  that  is  scheduled  for  implementation  in  two 
versions  or  “capabilities  packages”  (CPs). 

CP  1  is  scheduled  for  release  in  December  1996.  CP  2  is  scheduled  for  release  in 
April  1998  and  will  provide  embedded  instruction  and  help  features. 
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Service: 


Joint 


ET  System  Name: 

Parent  System:  Joint  Combat  Information  Terminal  (JCIT) 

Functional  Description 

JCIT,  under  development  by  the  Naval  Research  Laboratory  (NRL),  is  an  aviator 
commander’s  C2  system.  There  are  plans  to  include  ET  features  into  JCIT,  including 
embedded  instruction,  help  features,  and  extensive  built-in  test  (BIT)  capabilities  to  diag¬ 
nose  faults  to  a  replaceable  assembly  and  to  guide  the  user  in  the  replacement  procedure. 
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Service:  Navy 

ET  System  Name:  Video  Signal  Simulator  (VSS) 

Parent  System:  Navy  Tactical  Data  System  (NTDS) 

Functional  Description 

VSS  is  a  strap-on  signal  generator  for  simulating  radar  signals  on  the  NTDS  con¬ 
sole. 

Assessment 

“Generally,  the  system  users  indicated  that  the  VSS  was  a  very  good  training  sys¬ 
tem  when  it  was  working  correctly”  (Hoskin  et  al.,  1989,  p.  24). 

Positive  Features 

•  Can  be  used  both  in  port  and  at  sea,  although  it  is  used  more  often  at  sea. 

•  Users  can  create  scenarios  to  their  own  liking  (a  popular  feature). 

•  Simulated  signals  can  be  mixed  with  real-world  video  when  ship  is  underway; 
this  feature  is  not  often  used  because  of  safety  considerations. 

•  VSS  interfaces  with  weapons  systems  (to  an  extent). 

•  Scenarios  can  be  “frozen”  at  will. 

•  VSS  has  automatic  performance  measurement  capabilities.  Tracking  accuracy 
is  measured  and  stored  for  three  contacts;  performance  is  scored  on 
0-100  scale. 

Negative  Features 

•  Greatest  criticism  was  the  simulator’s  low-fidelity  simulation  of  generic  radar 
signals  with  limited  environmental  effects  (video  is  “too  clean”).  Other  specific 
fidelity  problems  include  the  lack  of  land-mass  simulation  (not  a  serious  prob¬ 
lem  at  sea)  and  simulation  of  jamming,  chaff,  and  cloud  formation  effects. 

•  Second  major  criticism  was  the  inability  of  the  system  to  interface  with  all  ele¬ 
ments  of  the  Combat  Information  Center  (CIC),  particularly  sonar,  EW,  and 
Automatic  Detector  Tracker  (ADT)  systems.  The  result  is  that  the  CIC  cannot 
be  realistically  exercised. 

•  Simulation  of  contact  is  limited  to  only  three  turn  rates. 

•  There  are  no  “canned”  scenarios.  Users  must  enter  data  to  initiate  each  sce¬ 
nario  run.  This  was  a  general  complaint  of  users. 
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Service:  Navy 

ET  System  Name:  Combat  Simulation  Training  [formerly.  Test]  System  (CSTS) 

Parent  System:  Radar  and  Fire  Control  Systems  (FCSs)  on  DDG-933  guided  mis¬ 

sile  destroyers 

Functional  Description 

The  previous  version  of  CSTS  was  a  strap-on  system,  which  was  stationed  pierside 
and  connected  to  the  ship  by  cables.  The  present  system  is  “fully  embedded”  and  can  be 
used  on-  or  off-line  while  the  ship  is  at  sea.  It  is  a  dual-purpose  system,  providing  simula¬ 
tion  scenarios  and  a  test  function  for  sensor  station  displays  and  weapon  system  equip¬ 
ment  It  simulates  radar  images  and  “stimulates”  the  ship’s  sonar  system  to  provide  sonar 
images.  The  systems  that  interface  with  CSTS  include  Command  and  Decision  (C&D) 
System,  Radio  Communications  System  (RCS),  Surveillance  and  AIMS  System  (SAS), 
Gun  Fire  Control  System  (GFCS),  Missile  Fire  Control  System  (MFCS),  the  Underwater 
Fire  Control  System  (UFCS),  and  ships  log  and  gyro  systems. 

Assessment 

“The  [previous]  DDG-993  version  of  the  CSTS  was  found  to  have  serious  prob¬ 
lems  in  supporting  training  activities.  This  is  due  mostly  to  the  fact  that  the  system  was 
designed  as  a  test  set  and  not  as  a  training  system”  (Hoskin  et  al.,  1989,  p.  40). 

“The  first-generation  CSTS  provide  effective  training,  but  it  was  not  considered  a 
practical  trainer  because  it  took  a  full  day  to  set  up  and  check  out  the  device  and  several 
hours  to  disconnect”  (Warm  et  al.,  1988,  p.  24). 

The  newer  version  still  has  problems  (Warm  et  al.,  1988),  including  increased 
major  pieces  of  equipment  and  resulting  maintenance  requirements  and  lack  of  performance 
measurement  capabilities.  However,  “. . .  the  CSTS  is,  by  itself,  a  simple  system  to  oper¬ 
ate,  but  start-up  and  shutdown  times  are  over  one  minute”  (Warm  et  al.,  1988,  p.  1-9). 

Positive  Features 

•  New  version  runs  canned  scenarios.  It  also  provides  the  training  officer  the 
ability  to  author  custom-designed  scenarios  and  store  them  on  disk. 

Negative  Features 

•  The  earlier  version  of  CSTS  had  to  be  used  with  other  simulation  systems 
(e.g.,  VSS,  sonar  target  generators)  for  training  applications. 

•  The  earlier  version  did  not  have  any  instructional  support  features — ^that  is, 
performance  measurement  capabilities,  scenario  authoring  capabilities,  and 
freeze  or  replay  features. 
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•  The  newer  version  also  does  not  include  performance  measurement,  feedback, 
or  report  generation  capabilities  (although  some  mission  data  can  be  retrieved 
from  the  NTDS). 

•  The  newer  version  does  not  include  scenario  freeze,  replay,  or  fast-forward 
features. 

•  Operating  the  newer  version  of  CSTS  precludes  consoles  selected  for  training 
from  performing  operational  functions. 

•  In  newer  version,  entities  simulated  on  Link-4  and  Link- 11  communications 
networks  are  not  communicated  to  other  consoles. 

•  New  version  of  CSTS  has  suffered  from  logistics  and  maintenance  problems. 
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Service:  Navy 

ET  System  Name:  AEGIS  Combat  Training  System  (ACTS) 

Parent  System:  AEGIS  Weapon  System  on  CG-47  class  cruisers 

Functional  Description 

ACTS,  which  is  fully  embedded  in  the  AEGIS  system,  presents  simulation  scenar¬ 
ios  for  training  functions  related  to  the  C&D  System,  Weapons  Control  System  (WCS), 
and  PCS.  ACTS  sends  data  directly  to  the  AN/SPY-1  radar.  Within  ACTS  is  the 
Computer-Aided  Submode  Training  (CAST)  system,  which  provides  computer-based  tuto¬ 
rials  for  initial  training  of  AEGIS  operators. 

Assessment 

“Reports  obtained  from  Fleet  users  indicate  that  ACTS  is  a  highly  effective  and 
desirable  means  of  training  .  .  .  Overall,  the  ACTS  capability  is  a  very  good  example  of 
what  an  ET  system  should  attempt  to  achieve”  (Hoskin  et  al.,  1989,  p.  39). 

Positive  Features 

•  ACTS  can  mixed  simulated  input  with  raw  real-world  video. 

•  ACTS  has  a  variety  of  canned  scenarios,  which  allows  flexibility  in  content 
and  size  of  team  being  trained. 

•  Canned  scenarios  can  be  altered  manually  on-line  (but  alterations  cannot  be 
saved). 

•  ACTS  can  generate  50+  surface  and  air  contacts. 

•  ACTS  trains  all  CIC  stations. 

•  ACTS  can  be  used  for  multi-ship  training  by  transmitting  symbology  and  raw 
video  to  other  participants. 

•  ACTS  has  capability  to  printout  overall  team  performance  in  terms  of  missile 
accuracy  and  kills  sustained  by  ownship. 

•  ACTS  has  scenario  playback  feature  (but  no  freeze  or  fast-forward). 

•  ACTS  start-up  time  (5  minutes)  and  shutdown  time  (less  than  1  minute)  are 
moderate. 

•  CAST  lessons  are  hierarchically  arranged. 

•  CAST  includes  performance  measurement  in  form  of  end-of-lesson  tests. 

•  CAST  maintains  student  records. 
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Negative  Features 

•  CAST  cannot  mix  real-world  and  simulated  video. 

•  Use  of  CAST  degrades  system  performance  by  taking  consoles  out  of  opera¬ 
tion  for  training. 

•  ACTS  has  no  scenario  freeze  and  fast-forward  capabilities. 

•  CAST  does  not  include  help  features. 

•  Users  see  CAST  as  suitable  only  for  initial  training. 

•  Custom-designed  ACTS  scenarios  cannot  be  saved. 

•  ACTS  has  no  automated  performance  measurement  or  trainee  record  keeping 
system. 
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Service:  Navy 

ET  System  Name:  Automatic  Detection  Tracking  System  Simulation  (ADSIM) 

Parent  System:  Automatic  Detector  Tractor  (ADT)  and  Weapons  Direction  System 

(WDS)  on  surface  ships 

Functional  Description 

ADSIM,  which  is  designed  for  both  weapon  system  tests  and  training,  uses  simu¬ 
lated  air  radar  targets  from  ADT  to  train  weapons  coordinators  and  weapons  system  teams. 
It  allows  “full  integration”  of  ADT  with  the  WDS. 

Assessment 

“Generally,  users  felt  that  the  system  provided  adequate  training  for  the  individuals 
and  subtearas”  (Hoskin  et  al.,  1989,  p.  37). 

Positive  Features 

•  ADSIM  provides  both  canned  scenarios  and  the  ability  to  author  (but  not  to 
store)  scenarios. 

•  Canned  scenarios  are  ordered  by  difficulty  to  adapt  to  entry-level  and  more 
advanced  operators. 

Negative  Features 

•  Video  has  minimal  realism  with  no  jamming  or  interference  simulation  or  other 
environmental  conditions.  Users  felt  the  screen  was  “too  clean.” 

•  System  cannot  be  linked  with  the  VS  S  to  provide  full  CIC  training. 

•  ADSIM  cannot  store  locally  authored  scenarios. 
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Service:  Navy 

ET  System  Name:  AN/BQR-T4  Sonar  Target  Simulator 

Parent  System:  AN/BQR-21,  -20,  -19,  -15  sonar  and  fire  control  systems  of  ballis¬ 

tic  missile  and  attack  submarines 

Functional  Description 

This  ET  system  generates  targets  by  stimulating  the  front-end  of  the  sonar  system 
just  after  the  hydrophone.  The  AN/BQR-T4  models  41  contacts  and  has  an  accompanying 
tape  mechanism  that  simulates  sonar  acoustics. 

Assessment 

“Generally,  users  indicate  that  the  BQR-T4  is  a  fairly  effective  and  reliable  mecha¬ 
nism  for  training.  Realism  for  the  system  was  deemed  adequate  for  most  training,  but 
could  use  some  improvements”  (Hoskin  et  al.,  1989,  p.  35). 

Positive  Features 

•  Users  are  most  pleased  with  ability  of  system  to  integrate  other  tactical  and 
weapons  stations  for  full-team  training. 

•  It  allows  adjustments  to  contact  intensity,  video  fading,  and  controllability  of 
contacts  throughout  the  scenario. 

•  System  allows  freedom  to  custom-design  scenarios  but  not  to  store  them. 

•  It  simulates  environmental  conditions  by  including  the  characteristics  of  seven 
different  oceans. 

•  BQR-T4  allows  the  mix  of  real  world  with  simulated  inputs  and  contacts. 

•  System  can  receive  ownship  movement  information  or  create  that  information 
to  simulate  movement 

•  BQR-T4  can  freeze  and  restart  scenario  to  provide  instruction  and  feedback. 

Negative  Features 

•  Principal  criticism  is  that  system  does  not  simulate  effects  of  ownship  motion 
on  target  motion  and  bearing,  thus  limiting  ability  to  drive  ownship  from  sonar 
and  to  mix  real  world  with  simulated  targets. 

•  Using  system  ties  up  a  few  sonar  consoles,  limiting  operational  capabilities 
somewhat. 

•  There  are  limitations  on  the  maximum  number  of  simultaneous  contacts. 

•  No  canned  (preprogrammed)  scenarios  are  provided. 


D-45 


•  Active  contacts  must  be  constantly  manipulated  to  achieve  acceptable  realism; 
system  should  be  automated. 

•  Users  have  identified  a  need  to  simulate  additional  environmental  effects  on 
sonar  characteristics,  including  bottom  type,  temperature,  depth,  time  of  year, 
and  layer  effects. 

•  System  does  not  simulate  weapons  firing  effects  to  permit  extended  operational 
training. 

•  System  does  not  have  performance  measurement  capabilities.  In  particular, 
users  want  a  record/replay  feature. 


D-46 


Service;  Navy 

ET  System  Name:  AN/S QS-56  Target  Simulator 

Parent  System:  AN/SQS-56  receiver  and  console  on  FFG-1  and  FFG-7  class 

frigates 

Functional  Description 

The  AN/SQS-56  is  a  fully  embedded  system  that  injects  simulated  sonar  threat  tar¬ 
gets  into  sonar  consoles.  It  is  often  used  with  text-based  Operational  Readiness  and 
Assessment  Training  System  (ORATS)  to  create  training  scenarios. 

Assessment 

Results  indicate  that  the  AN/SQS-56  simulation  capability  is  an  effective  means  for 
training,  particularly  when  combined  with  ORATS  [Operational  Readiness  Assessment  and 
Training  System]  lessons  and  the  accompanying  performance  measurement  guides” 
(Hoskin  et  al.,  1989,  p.  34). 

Positive  Features 

•  System  can  mix  simulated  with  real-world  targets,  which  is  viewed  as  desir¬ 
able  and  not  a  safety  hazard. 

•  System  can  simulate  fades  going  below  the  layers  with  a  variable  contact  inten¬ 
sity  adjustment. 

•  System  is  effectively  integrated  with  fire  control  and  NTDS,  providing  full 
team 

Negative  Features 

•  Simulator  does  not  have  any  preprogrammed  scenarios. 

•  High-fidelity  simulation  effects  require  extensive  data  inputs  and  manipulations 
of  contact  characteristics  during  scenarios. 

•  Simulation  of  NTDS  inputs  limited  to  contact  symbology  (i.e.,  no  video). 

•  Inability  to  include  passive  acoustics  was  major  criticism. 

•  There  are  no  preprogrammed  or  “canned”  scenarios,  and  there  is  no  capability 
to  custom  design  scenarios  to  needs. 
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Service; 


Navy 


ET  System  Name:  In-Flight  Trainer  (IFT) 

Parent  System:  F- 14  aircraft 

Functional  Description 

The  IFT  is  software  embedded  in  the  F-14’s  central  computer.  It  presents  simula¬ 
tion  scenarios  to  train  pilots  and  radar  intercept  officers  to  deploy  missiles  against  radar 
contacts.  (No  gatling  gun  tasks  are  trained.)  IFT  uses  aircraft  systems,  with  no  additional 
equipment  required.  Scenarios  are  stored  in  read  only  memory  (ROM)  and  present  targets 
flying  on  preprogrammed  paths.  Synthetic  targets  are  principally  presented  on  the  radar 
intercept  officer  (RIO)  screen. 

Assessment 

The  F-14  is  a  reliable  system  that  is  usually  available  for  training.  “Operating  the 
IFT  is  very  simple,  requiring  minimal  effort  on  the  part  of  the  crew  members.  The  IFT  ini¬ 
tializes  and  shuts  down  almost  instantaneously”  (Warm  et  al.,  1988,  p.  G-4). 

Positive  Features 

•  IFT  simulates  radar,  identification  of  friend  or  foe  (IFF),  ECM,  and  ECCM 
equipment  with  fairly  high  fidelity. 

•  Activation  of  IFT  is  virtually  instantaneous,  requiring  one  switch  action  by 
pilot  and  three  by  the  RIO. 

•  IFT  uses  canned  scenarios,  which  can  be  randomly  selected  by  system,  or  the 
crew  can  select  specific  scenarios. 

•  System  provides  hit/miss  data  to  crew. 

Negative  Features 

•  Scenarios  are  stored  in  ROM  and  cannot  be  authored.  ROM-based  scenarios 
are  difficult  to  update — limited  to  only  major  software  upgrades. 

•  Performance  data  cannot  be  stored  on  magnetic  media  or  printed  out  on  paper; 
this  precludes  use  of  debrief  site. 

•  System  has  no  scenario  freeze,  replay,  or  fast-forward  capabilities. 

•  IFT  caimot  send  simulated  targets  over  commo  net  for  coordinate  training  with 
other  units. 

•  (As  of  1988)  IFT’s  lack  of  memory  limits  the  number  of  scenarios  that  can  be 
stored  and  their  complexity. 

•  Fidelity  does  not  permit  training  of  air-to-air  gunnery  with  gatling  gun. 
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Service:  Navy 

ET  System  Name:  Performance  Measurement  and  Evaluation  (PME)  for  Sonar 
Systems 

Parent  System:  Sonar  systems  on  surface  ships 

Functional  Description 

PME  is  a  strap-on  tape  recorder  system  that  records  and  plays  back  sonar  contacts 
(video),  accompanying  acoustics,  and  voice  annotations  of  real-world  sonar  operations. 
These  tapes  are  normally  evaluated  by  shore-based  experts. 

Assessment 

“Results  indicate  that  PME  as  a  stand-alone  means  of  training  is  basically  ineffective 
.  .  .  [but  if  used  with  a  target  generator].  .  .,  it  provides  an  extremely  useful  capabil- 
ity”(Hoskin  et  al.,  1989,  p.  33). 

Positive  Features 

•  PME  provides  an  excellent  means  for  evaluating  performance. 

Negative  Features 

•  Use  is  limited  by  the  fact  that  playback  is  through  the  sonar  console,  thereby 
eliminating  all  operational  capabilities.  This  limits  its  usefulness  at  sea. 

•  PME  is  not  sufficient  for  presenting  training.  At  a  minimum,  it  must  be  used 
in  conjunction  with  target  generators. 
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Service:  Navy 

ET  System  Name:  Unit  34,  Sonar  Target  Generator 

Parent  System:  AN/SQS-53  sonar  consoles  and  NTDS  consoles  on  surface  ships 

Functional  Description 

Unit  34  is  a  strap-on  system  for  generating  sonar  targets.  It  injects  sonar  contacts 
(stimulates),  sonar  consoles,  and  NTDS  consoles.  Signals  can  vary  in  intensity  and  noise 
be  imbedded  in  many  environmental  conditions. 

Assessment 

“Generally,  Unit  34  is  judged  to  be  a  fairly  adequate  training  device  for  sonar 
operations”  (Hoskin  et  al.,  1989,  p.  32). 

Positive  Features 

•  Controllability  of  contacts  and  ability  to  simulate  environmental  conditions  are 
highly  regarded  aspects  of  system. 

•  Scenarios  can  be  authored  at  the  initiation  of  training. 

•  Scenarios  can  be  frozen. 

Negative  Features 

•  Complex  enviroranental  conditions  can  only  be  simulated  by  constant  manual 
inputs.  System  would  benefit  from  automation  of  conditions. 

•  There  are  no  canned  scenarios.  All  scenarios  must  be  pre-planned  and  input 
into  the  system. 

•  Freezing  scenarios  can  lead  to  unexpected  results  (e.g.,  restarting  the  sce¬ 
nario). 

•  Each  console  used  by  Unit  34  reduces  the  operational  capabilities  of  the  sys¬ 
tem. 

•  The  system  is  unable  to  mix  real-world  and  simulated  video  for  training  at  sea. 

•  Unit  34  does  not  interface  with  Mk  116  FCS  that  would  allow  team  training  of 
fire  control. 

•  Unit  34  has  no  performance  measurement  capabilities. 
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Service:  Navy 

ET  System  Name:  Operational  Training  Software  (OTS) 

Parent  System:  AN/SLQ-32  console  on  surface  ships 

Functional  Description 

OTS  simulates  EW  signals  through  the  tape  drive  on  the  AN/SLQ-32.  The  system 
includes  a  total  of  three  pre-recorded  tapes,  each  containing  three  simulation  scenarios. 

Assessment 

“Results  of  the  analysis  with  system  users  indicate  that  the  OTS  is  an  average-to- 
poor  means  of  training  EW  operations”  (Hoskin  et  al.,  1989,  p.  29). 

Positive  Features 

•  Tapes  can  be  played  at  lOX  as  well  as  IX  speed  for  scenario  presentation. 
Negative  Features 

•  OTS  requires  the  same  console  and  tape  drive  as  operational  system;  thus,  it 
requires  that  the  SLQ-32  to  be  taken  out  of  operational  status.  This  severely 
restricts  the  usability  of  the  system. 

•  Training  tapes  have  not  been  revised  in  accordance  with  operational  upgrades. 

•  OTS  does  not  interact  with  any  other  system,  as  the  AN/SLQ-32  would  in  the 
operational  mode.  In  particular,  the  system  should  interact  with  NTDS  light 
Airborne  Multi-Purpose  System  (LAMPS)  Mk  III,  electroiuc  support  measure 
(ESM),  and  the  EW  supervisor  station  to  provide  EW  countermeasures. 

•  The  simulations  low  fidelity  is  judged  inadequate  for  training  experienced 
operators.  The  simulation  does  not  include  friendly  signatures  or  noise  or 
audio/acoustics  and  includes  only  a  few  hostile  signatures. 

•  There  is  no  capability  to  author  or  edit  the  “canned”  scenarios  on  tape. 

•  Scenarios  “frozen”  for  any  reason  must  be  restarted  at  the  beginning  of  the 
tape. 

•  OTS  has  no  performance  measurement  capabilities. 

•  OTS  has  no  capability  to  mix  live  and  simulated  targets. 


Service;  Navy 

ET  System  Name:  Sonar  Target  Signal  Simulator  (STSS) 

Parent  System:  AN/SQS-53,  -53A  sonar  consoles  on  anti-submarine  warfare 

(ASW)  ships  and  destroyers 

Functional  Description 

STSS  is  a  strap-on  device  for  simulating  the  visual  and  audio  input  to  train  sonar 
operators  and  teams  in  ASW  exercises. 

Assessment 

“Results  of  the  analysis  with  system  users  determined  that  the  STSS  is  not  judged 
to  be  a  usable  system .  .  .The  ships  that  had  STSS  did  not  use  it,  and  little  specific  infor¬ 
mation  on  its  capabilities  was  available”  (Hoskin  et  al.,  1989,  p.  27). 

Positive  Features 

•  Users  can  create  their  own  scenarios. 

•  STSS  can  simulate  some  weapons  being  fired  at  ship. 

•  STSS  can  include  some  environmental  effects. 

Negative  Features 

•  The  primary  objection  is  that  the  STSS  runs  much  too  fast  to  be  realistic. 

•  Operational  equipment  must  be  specially  aligned  for  the  STSS  training  run. 

•  ITiere  are  no  canned  scenarios.  All  scenarios  must  be  pre-planned  and  input 
into  the  system. 

•  Little  or  no  documentation  exists. 
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Service:  Navy 

ET  System  Name:  Training  Module 

Parent  System:  Joint  Maritime  Command  Information  System  (JMCIS)  and  the 

Navy  Tactical  Command  System  (NTCS) 

Functional  Description 

Under  development  by  NRL,  the  training  module  offers  the  following  capabilities: 
to  allow  the  trainer  to  author  ET  sessions  using  NRL’s  Presentation  Authoring  Language 
(PAL),  to  present  computer-based  tutorials  to  trainees,  to  present  dynamic  scenarios  to 
trainees,  and  to  create  and  present  sessions  that  integrate  tutorials  and  scenarios. 

The  ET  system  is  fully  embedded  in  that  software  and  calls  relevant  library  func¬ 
tions  within  the  parent  system.  Although  training  and  operational  computing  can  proceed 
simultaneously,  the  training  module  does  not  conflict  with  operational  data  or  processes. 
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Service:  Navy 

ET  System  Name:  Lesson  Translator  (L-TRAN) 

Parent  System:  Naval  Tactical  Data  System  (NTDS) 

Functional  Description 

L-TRAN  is  an  embedded  computer-based  tutorial  about  NTDS  consoles,  symbol¬ 
ogy,  and  operation.  It  provides  entry-level  and  sustainment  training  for  NTDS  operators 
on  Navy  carriers,  cruisers,  destroyers,  and  frigates. 

Assessment 

“Reports  obtained  from  Fleet  users  indicate  that  the  L-TRAN  is  a  highly  effective 
and  desirable  means  of  training”  (Hoskin  et  al.,  1989,  p.  22). 

Positive  Features 

•  Provides  individualized  instruction  that  requires  little  supervision. 

Negative  Features 

•  Requires  that  multiple  NTDS  computers  be  taken  out  of  operation.  A  mini¬ 
mum  of  4  (out  of  20)  consoles  are  required  for  training,  which  limits  use  of  ET 
at  sea. 

•  Initiation  of  ET  involves  tape  loading  and  manipulations  requiring  several 
minutes. 
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Service:  Navy 

ET  System  Name:  SPA-25G  Embedded  Training  System  (SETS) 

Parent  System:  AN/SPA-25G  Radar  Repeater  Console 

Functional  Description 

SPA-25G  is  a  microprocessor-based  system  that  can  display  any  radar  image  gen¬ 
erated  within  the  Radar  Display  and  Distribution  System  (RADDS).  SETS  is  a  proof-of- 
concept  demonstration  to  show  how  an  ET  system  would  be  used  with  this  console.  SETS 
is  made  up  of  three  components:  (1)  the  SPA-25G,  an  off-the-shelf  radar  signal  simulator 
(Buffalo  Computer  Graphics  RS-11),  (2)  a  Zenith  248  DOS-based  computer,  and  (3)  an 
interface  (RS-232)  between  the  computer  and  simulator.  SETS  is  designed  to  train  indi¬ 
vidual  operator  tasks  in  four  areas:  (1)  equipment  proficiency,  (2)  radar  navigation,  (3)  air 
intercept  control,  and  (4)  anti-submarine  air  control. 

Assessment 

The  system  has  not  been  formally  appraised,  but  Lacy,  Ellis,  and  Madden  (1990) 
offer  the  following  cost  analysis:  “Instructors  at  “C”  schools  for  air  controllers  estimate 
that  SETS  may  reduce  attrition  by  50  percent.  Given  that  the  throughput  at  the  school  is 
58  students  and  each  student  costs  $30K  to  train,  SETS  is  saving  $870K  per  year 
(29  X  $30K). 

Positive  Features 

•  SETS  has  the  capability  to  create  and  store  scenarios. 

•  System  records  and  manages  student  performance  data. 

•  During  replay,  the  students  and  the  instructor  can  compare  student  performance 
against  some  “ideal”  target  or  standard  that  can  be  preprogrammed  by  the 
instructor. 

•  SETS  has  capability  to  freeze,  replay,  and  play  at  faster  or  slower  than  real 
time. 

•  System  provides  printed  feedback  for  student. 

•  SETS  can  record  all  voice  dialogue  that  occurs  during  a  training  session. 

•  Scenario  generator  has  “rehearsal”  feature,  where  the  developer  can  view  the 
scenario  as  it  is  being  developed. 

Negative  Features 

•  The  system  does  not  have  voice  recognition  and  synthesis  capabilities. 
Therefore,  “pseudo-pilots”  are  needed  to  role  play  for  air  control  tasks. 
Pseudo-pilot  errors  make  evaluation  of  student  performance  more  difficult. 
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•  Scenario  generator  requires  the  input  of  detailed  data,  such  as  position, 
bearing,  and  velocity.  Graphical  authoring  features  that  allows  developers  to 
place  aircraft  directly  on  the  PPI  would  make  scenario  development  faster  and 
easier. 

•  The  RS-1 1  simulator  was  not  designed  for  air  control  tasks  and,  consequently, 
lacks  fidelity  in  that  regard. 
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Service: 


Navy 


ET  System  Name:  Interactive  Multisensor  Analysis  Training  (IMAT) 

Parent  System:  [Shipboard  anti-submarine  warfare  (ASW)  systems] 

Functional  Description 

The  current  version  of  IMAT  is  a  “scientific  simulation”  that  trains  sonar  technicians 
and  aviation  warfare  system  operators  in  the  fundamental  concepts  related  to  their  jobs.  It 
is  designed  to  facilitate  the  acquisition  of  high-level  cognitive  skills  that  are  normally 
acquired  through  extended  on-the-job  training.  Although  it  is  presendy  a  stand-alone 
simulation,  there  are  plans  to  develop  an  onboard  version,  which  would  include  high- 
fidelity  databases,  near-real-time  simulations,  and  visualizations  for  training  tactical  plan¬ 
ning  and  reconstruction.  The  onboard  version  extends  the  IMAT  concept  from  operator 
training  to  training  an  integrated  Commanding  Officer  (CO)/Executive  Officer  (XO)/Sonar 
team. 
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Service: 


Navy 

ET  System  Name:  Organic  Combat  System  Training  Technology  (OCSTT)  Simulation 
System 

Parent  System:  FFG-7  tactical  systems 

Functional  Description 

The  OCSTT  Simulation  System  is  a  technology  demonstration  designed  to  replace 
the  pierside  Device  20B5  with  modem  technology  that  is  substantially  cheaper  and  that 
converts  the  system  to  a  shipboard  training  device,  or  what  the  authors  term  an  “organic 
training  system.”  The  system  maintains  approximately  70  percent  of  the  older  device’s 
functionality. 

An  important  improvement  is  the  incorporation  of  DIS  capabilities.  The  target  for 
transitioning  this  technology  is  the  Navy’s  Battle  Force  Tactical  Training  (BFTT)  system. 
Hardware  costs  of  OCSTT  were  less  than  $200K,  compared  to  more  than  $10M  for  Device 
20B5. 
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Service:  Navy 

ET  System  Name:  Device  20B4/B5 

Parent  System:  FFG-7  and  other  surface  ships 

Functional  Description 

Built  by  AAI  Corporation,  the  20B4  and  20B5  are  umbilical  pierside  ET  systems 
that  can  be  connected  to  as  many  as  six  ships  for  simultaneous  training.  They  “stimulate” 
radar,  communications,  and  EW  and  ASW  receivers.  They  also  “simulate”  weapon  sys¬ 
tems  (guns,  torpedos,  missiles)  and  decoy  systems.  The  training  focus  is  on  team — ^rather 
than  individual  operator — training.  Both  systems  use  carry-on  boxes  and  umbilical  cords 
attached  to  a  van  parked  near  the  pier.  Whereas  the  20B5  supports  only  FFG-7  training, 
the  20B4  supports  training  on  other  types  of  ships  and  has  limited  support  for  AEGIS 
capabilities.  The  van  can  drive  DIS  scenarios  that  are  compatible  with  BFTT  training  per¬ 
formed  at  sea.  The  20B5  system  costs  in  excess  of  $10M. 

Assessment 

According  to  Stratton  et  al.  (1996),  the  20B5  “.  .  .  is  a  proven  training  system  and 
has  a  good  reputation  with  the  fleet .  .  .  The  interface  points  with  FFG-7  tactical  systems 
have  been  proven  to  work  many  times  over”  (p.  11). 
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Service;  Navy 

ET  System  Name:  Training  Alarm  Controller  (TAC) 

Parent  System:  United  Kingdom  Polaris  Fire  Control  System  (FCS) 

Functional  Description 

TAC  is  a  fully  embedded  system  that  simulates  failures  in  the  Polaris  FCS.  To  use 
the  TAC,  the  FCS  must  be  operating  so  that  the  gyros  are  energized  and  signals  are 
received  from  the  guidance  system.  Faults  are  simulated  by  controlling  the  relays  that 
would  be  activated  in  the  event  of  a  real  fault — a.  “stimulation”  of  sorts.  The  TAC  is  used 
to  train  the  Weapon  Engineer  Officer  (WEO)  and  his  team  in  procedures  related  to  failures 
in  the  countdown  and  launch  sequence.  TAC  is  usually  used  with  the  Patrol  Analysis 
Recording  System  (PARS),  which  is  a  tape  recorder  for  recording  communications  and  pen 
event  recorders. 

Assessment 

“The  consensus  is  that,  despite  the  shortcomings  [listed  below],  TAC  is  a  valuable 
item  of  equipment  However,  due  to  the  way  it  is  used,  no  quantitative  data  are  available 
on  its  actual  effectiveness  in  imparting  and  maintaining  skills”  (Annett,  1990,  p.  9).  TAC 
is  used  with  PARS  during  Weapon  Systems  Readiness  Tests  (WSRTs),  which  occur  at 
irregular  intervals  but  average  about  once  per  week  during  the  submarine  patrol.  Results 
are  not  given  to  crew;  rather,  they  are  provided  to  the  Ministry  of  Defence.  Less  formal 
exercises  are  performed  by  the  weapon  systems  crew  during  Independent  Exercises 
(INDEX)  which  occur  at  sea  in  preparation  for  patrol.  The  WEO  maintains  these  records 
for  training  purposes. 

Positive  Features 

•  FCS  shows  indicator  lights  operating  as  they  would  with  an  actual  fault 
Although  the  TAC  offers  less  fidelity  than  shore-based  tactical  simulators,  it 
has  been  estimated  that  70  percent  of  the  faults  occurring  during  missile  launch 
countdown  can  be  reproduced  on  TAC  at  sea. 

Negative  Features 

•  Location  of  TAC  is  inconvenient  and  limits  its  use,  and  surprise  faults  are  vir¬ 
tually  impossible.  Access  to  the  system  is  tolerably  inconvenient  for  the  WEO 
but  intolerably  inconvenient  for  an  additional  “sea  riding”  evaluator. 

•  It  does  not  fully  train  or  test  the  entire  team.  Training  is  limited  to  the  weapon 
system  team  and,  during  INDEX,  does  not  include  the  WEO  who  conducts  the 
training  and  evaluates  his  team. 

•  The  PARS  system  is  used  only  for  WSRTs,  not  for  individual  feedback. 
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•  Team  members  can  see  fault  being  inserted  into  the  system.  The  type  of  fault 
is  hidden  by  a  metal  flap,  but  the  fact  that  a  fault  is  being  input  cannot  be 
hidden. 

•  Some  power  fault  indications  cannot  be  simulated  by  TAC. 

•  "niere  are  fears  that  TAC  contributes  wear-and-tear  on  the  PCS  relays. 
Specifically,  it  may  induce  false  positive  indications  caused  by  a  sticking  relay. 

•  Forgetting  to  reset  fault  indicator  may  lead  to  a  false  positive  indication, 
although  this  can  easily  be  avoided. 

•  Technical  documentation  is  outdated  and  inconvenient  to  access.  It  is  buried  in 
longer  documents  on  operation  of  the  PCS. 
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Service:  Navy 

ET  System  Name:  Sonar  Operator  Trainer  14E35  SN  2/3, 14E35C  SN  4,  and  14E35C 
SN5 

Parent  System:  Sonar  Systems:  AN/SQQ-89(V)l/2,  AN/SQQ-89(V)2/4,  and 

AN/SQQ-89(V)6/9 

Functional  Description 

Located  at  the  Fleet  Anti-Submarine  Warfare  Training  Center  Pacific,  this  embed¬ 
ded  stimulation  systems  is  used  in  conjunction  with  SQQ-89  Interactive  Courseware  (ICW) 
training  devices  built  by  NAVSEALOGCENDET.  These  trainers  teach  sailors  in  the  Sonar 
Technician  (ST)  rate  and  are  very  expensive,  high-fidelity  devices  that  use  a  combination  of 
shipboard  equipment  and  simulation/stimulation  equipment. 
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Service:  Navy 

ET  System  Name:  Electronic  Warfare  Intelligent  Embedded  Training  (EWIET) 

Parent  System:  SLQ-32  Electronic  Warfare  (EW)  console 

Functional  Description 

EWIET  is  a  proof-of-concept  demonstration;  that  is,  it  is  not  yet  embedded  in  the 
operational  system.  It  is  currently  implemented  on  a  computer  simulation  of  the  SLQ-32 
display.  The  process  control  architecture  for  the  EWIET  is  the  Knowledgeable  Observation 
Analysis-Linked  Advisory  System  (KOALAS),  which  models  the  hmnan’s  tactical  situa¬ 
tion  assessment  and  provides  context  for  sensor  fusion  systems  to  initialize  and  maintain 
this  assessment. 
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Service:  Navy 

ET  System  Name:  Advanced  Embedded  Training  (AET) 

Parent  System:  AEGIS  Weapon  System  on  CG-47  class  cruisers 

Functional  Description 

AET — ^a  research  and  development  (R&D)  6.2  project  for  the  Naval  Air  Warfare 
Center  (NAWC) — is  part  of  an  Advanced  Technology  Demonstration  (ATD)  to  integrate 
human  performance  technologies  in  AEGIS  Combat  System  and  Combat  Training  System. 
The  prototype  will  be  designed  to  train  the  CO,  the  tactical  action  officer  (TAO),  the  combat 
systems  coordinator  (CSC),  the  anti-aircraft  warfare  coordinator  (AAWC),  the  tactical 
information  coordinator  (TIC),  the  identification  coordinator  (IDC),  and  the  electronic 
warfare  supervisor  (EWS).  It  will  be  supported  by  a  training  supervisor  or  scenario 
manager.  Tlie  eventual  product  will  provide  an  adjunct  to  the  ACTS. 

Assessment 

A  preliminary  estimate  by  system  developers  was  that  the  “.  .  .  net  result  of  this 
AID  for  Ae  fleet  will  be  a  vastly  improved  capability  for  ships  to  independently  conduct 
comprehensive,  consistent,  timely,  and  effective  team  training  in-port  and  at-sea  ...  the 
AET  system  will  significantly  reduce  the  requirements  for  assignment  of  Afloat  Training 
Organizations  and  AEGIS  Training  Support  Group  training  support  personnel  to  ships, 
which  could  result  in  a  $1.5M/year  cost  savings.  Moreover,  the  expected  level  of  training 
enhancement  is  a  25%  to  40%  step  improvement  in  team  performance  as  determined  by 
validated  measures  of  effectiveness”  (NAWC,  1996). 

Positive  Features 

•  AET  will  provide  operator  and  team  performance  measurement. 

•  AET  will  provide  diagnosis/assessment  of  performance. 

•  AET  will  provide  conective/facilitative  feedback  and  computer-assisted 
coaching. 
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Service: 


Navy 


ET  System  Name:  Radar  Electromagnetic  Environmental  Simulator  (REES-204) 
Parent  System:  AN/SPS-48  radar  console  on  surface  ships 

Functional  Description 

REES  is  a  built-in  system  that  generates  simulated  radar  targets  to  train  operators 
and  teams  on  systems  in  New  TTireat  Upgrade  Ships.  The  REES  signal  stimulates  the 
radar  receiver  and  can  be  received  by  any  console  that  gets  SPS-48  input.  The  REES  is 
scheduled  to  be  replaced  by  the  Radar  Environment  Simulator  System  (RESS). 

Assessment 

‘The  REES  System  was  evaluated  by  users  as  a  fairly  effective  means  of  training 
radar  operators  and  teams”  (Hoskin  et  al.,  1989,  p.  28). 

Positive  Features 

•  Ease  of  use  and  system  initiation  were  often  cited  as  key  determinants  of  REES 
usefulness. 

•  Number  of  targets  (9)  and  controllability  of  these  targets  were  viewed  as  desir¬ 
able  features. 

•  It  allows  mixing  of  real-world  and  simulated  video,  thereby  increasing  its 
usability  at  sea. 

•  It  is  a  fairly  high  fidelity  system,  accurately  simulating  acceleration/deceleration 
rates  and  jamming,  and  emulates  some  environmental  noise. 

•  Users  can  create  their  own  scenarios. 

•  Trainers  can  freeze,  slow,  and  speed  up  scenarios. 

Negative  Features 

•  Greatest  criticism  is  that  it  does  not  interface  with  IFF,  missile  control,  and 
other  radar  systems  to  allow  training  of  the  Combat  Information  Center  (CIC) 
team. 

•  No  documentation  exists.  The  users  were  personally  trained  by  contractors. 

•  Simulation  lacks  chaff  and  other  radar  sources  of  interference. 

•  The  system  does  not  have  capability  to  create  and  reuse  “canned”  scenarios. 

•  The  system  has  no  performance  measurement  capabilities,  although  users 
viewed  this  only  as  a  nonessential  (“nice  to  have”)  feature. 
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